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Executive Summary 



This report presents an evaluation of the Enhancing Education Through Technology (Ed-Tech) 
Program in Vermont, with a focus on the state’s competitive grants. The Ed-Tech Program is a 
funding source authorized under Title II, Part D, of the No Child Left Behind (NCLB) Act of 
2002. In 2009, Ed-Tech funds allocated to Vermont were supplemented by funds from the 
American Reinvestment and Recovery Act (ARRA). The Ed-Tech Program is administered by 
the U.S. Department of Education through its Office of School Support and Technology 
Programs. Funding available under the Ed-Tech Program supports NCLB goals in the following 
ways: 

• By closing the achievement gap by providing access to smart computing devices, digital 
content, and open education resources for all students via 21st century learning 
environments that enhance teaching and learning with technology integration 

• By supporting the development of highly qualified teachers with online courses, 
communities of practice, instructional technology coaches, and virtual communication to 
ensure flexibility and access 

• By using data for school improvement and individualizing instruction for all students 

Evaluation Overview 

American Institutes for Research (AIR) conducted an external evaluation of Vermont’s Ed-Tech 
Program from January 2010 - June 2011. The purpose of the evaluation was to provide feedback 
on the implementation and outcomes of the competitive grant programs that were active during 
this time. The findings of the evaluation are intended to help VTDOE learn from successful 
components of the grant initiatives — so these components can be continued or scaled up — and 
about less successful components — so these activities can be improved or funds redirected. This 
report addresses the competitive grant programs that continued or commenced during the 2010- 
2011 school year. 

During the planning stage of the evaluation, the AIR evaluation team worked with VTDOE and 
the coordinators of the different competitive grant programs to specify a set of evaluation 
questions. On the basis of this information, the evaluation addressed the following questions: 

1 . To what extent and with what fidelity are the grantees of the Ed-Tech Program making 
progress toward their stated objectives? What has facilitated or hindered progress? 

la. To what extent have grant funds been spent so far and on what? 

lb. What technology resources have been and will be purchased, distributed, and 
supported? 

2. How effectively do schools support the implementation of project goals? 

2a. What types of training and professional development were offered? 

2b. What is the scope of participation in professional development? 
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2c. What are participants’ opinions of the quality and the effectiveness of the 
professional development? 

2d. To what extent are teachers provided opportunities to collaborate on implementing 
program objectives for technology integration? 

2e. To what extent do administrators support, advocate, and encourage technology 
integration? 

2f. To what extent and from what sources do teachers receive technology support? 

3. Do the Ed-Tech-funded competitive grant programs promote technology integration in 
support of student-centered learning? 

4. What are the learning outcomes of the program in terms of student engagement and 
motivation and mastery of Vermont grade-level expectations? 

5. To what extent are changes in teaching and learning practices adopted and sustained, as 
indicated by continued and expanded use of such practices by teachers and school leaders 
who took part in the program? 

6. To what extent did the program support the development and the deployment of online 
learning opportunities? 

The following six competitive grant programs were included in the scope of the evaluation: 

• Content-Based Technology Grants (CBTG). Fifty-two CBTG grants were awarded to 
schools with the goal of providing modest financial assistance to schools with high 
poverty and the greatest need for technology support, including schools identified for 
improvement under NCLB. Through the grants, schools developed programs to integrate 
technology and equipment into schools based on specific content areas. 

• The Educate/Innovate Program made grants to 17 schools or supervisory unions to 
incorporate various types of technology into teaching and learning in the classroom. The 
funds from the grants were intended to support innovative practices using technology to 
support learning. 

• The eLearning Project provided professional development services to six demonstration 
sites that will serve as a model for other schools in making the transition to 2 1 st century 
learning in Vermont, as well as building community engagement. 

• The Learning Network of Vermont (LNV) is a video-conferencing network, currently 
operating in 104 elementary, middle, and high schools in Vennont with the goal of 
promoting flexible learning environments for students. 

• The Vermont Virtual Learning Cooperative (VTVLC) provides K-12 online courses 
in a wide variety of subject areas to schools around the state. 

• Real World Design Challenge (RWDC) is a competition involving teams of high school 
students who attempt to solve an engineering challenge currently faced by industry. 
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Evaluation Methods 



The evaluation collected data from surveys and interviews with participating teachers, grant 
managers, principals and other key informants. The following is a summary of the topics 
addressed by each data collection method and the type of respondent: 

Teacher Survey. The teacher survey addressed the topics with which teachers had direct 
experience, including professional development, teacher collaboration, administrative direction 
and support, impact on teacher practice, and student outcomes. Teachers associated with five of 
the six programs were invited to respond to the survey; teachers participating in VTVLC were 
not invited because the topics covered by the survey were not related to the VTVLC program. 

Grant Manager Survey. The grant manager survey addressed the evaluation questions grant 
managers were most knowledgeable about, which included the use of grant funds and program 
participation. Some programs had several grant managers (e.g., CBTG, Educate/Innovate) and 
some had a single grant manager (e.g., eLearning, LNV). The VTVLC grant manager was not 
sent this survey because the items addressed technology integration programs, rather than online 
learning efforts. 

Survey response frequencies were examined in the aggregate and by program. Where response 
frequencies differed notably by program, the survey findings are presented for each program. 
Where there is little variation by program, the response frequencies are presented in aggregate. 

Interviews and site visits. The interview methodology involved selecting up to four sites for 
each program and conducting a series of interviews at each site. Typically, interviews were 
conducted with participating teachers, the grant manager, and the school principal. For some 
programs, additional types of interviews were conducted. Site visits were conducted in fall 2010 
and spring 2011. 

The analysis of the interviews employed a two-step process. First, the evaluator developed site 
profiles based on the responses of all individuals from each school. These profiles were 
organized according to the topics included in the evaluation questions. Second, a cross-site 
analysis was conducted for each program to identify themes across school profiles. 

Conclusions 

Question 1: To What Extent and With What Fidelity Are the Grantees Making Progress 
Toward Their Objectives? 

Overall, grantees have made significant progress toward their objectives and are largely 
implementing their programs with fidelity. This is observed in relation to the foci of grantees, 
their spending of grant funds, and in terms of the technology resources acquired through the 
grant. To begin with, the foci of the grant programs were well aligned with the statewide Ed- 
Tech Program goals. In particular, most grant managers indicated that technology integration and 
the promotion of 21st century skills were the major goals of their grants. The grants frequently 
focused on professional development and teacher collaboration, both of which are strategies of 
the statewide Ed-Tech Program. 
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The progress of grantees is also indicated by survey responses regarding the spending of grant 
funds. Grant managers typically reported having spent the majority of grant funds overall, as 
well as those funds allocated to professional development. Grantees are acquiring the technology 
resources that are expected by the program. 

Question 2: How Effectively Do Schools Support the Implementation 
of Project Goals? 

Professional development was considered by teachers to be of high quality and effective. The 
vast majority of teachers rated professional development as high quality, and nearly 75 percent 
of teachers across grant programs reported that their participation had a noticeable to very strong 
impact on their teaching practice. Teachers indicated that professional development was highly 
targeted and relevant. The professional development in these programs appears to have been 
sustained — that is, occurring over several months. The eLearning Project teachers reported 
particularly high levels of impact on teaching practice. This may reflect the program’s extensive 
coaching, team-based approach, and targeting of specific school goals. 

The Ed-Tech Program enhanced collaboration, which in turn supported implementation of the 
grant goals. Grant managers and teachers indicated that teacher collaboration was a considerable 
focus of their Ed-Tech grants. Teachers reported that collaboration played a very important role 
in achieving the goals of their grant. 

School leadership actively supported the grant goals with resources and technical support. 
Teachers reported high levels of principal awareness and encouragement of technology 
integration related to the grant. School leaders supported grants by providing resources (e.g., 
planning time) as well as leadership (e.g., setting expectations for technology integration). 
Teachers from the eLearning Project reported the highest level of principal involvement. In other 
programs, school and district technology leaders provided guidance and support. Teachers 
typically reported receiving sufficient technical support to accomplish the goals of their grant 
programs. 

Question 3: Do the Ed-Tech Grant Programs Promote Technology Integration in Support 
of Student-Centered Learning? 

The Ed-Tech Program appears to have promoted student-centered instruction. The majority of 
teachers reported that their Ed-Tech grant programs had a noticeable to very strong impact on 
instructional practices, with the exception of LNV teachers. The programs promoted student- 
centered instruction in a variety of ways, including more project-based, collaborative, and 
personalized learning and greater differentiation of instruction. Across all programs, teachers 
noted that they assigned more projects than in the previous school year. 

In the opinion of participants, the Ed-Tech Program increased the frequency of student 
technology use. Overall, teachers indicated that they are integrating technology into the 
classroom on a more frequent and consistent basis, particularly when compared to the previous 
school year. Teachers reported frequent student use of technology for several purposes, including 
additional practice or skill reinforcement, online research, and paper writing. 
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Question 4: What Are the Learning Outcomes of the Program in Terms of Student 
Engagement and Motivation and Mastery of Vermont Grade-Level Expectations? 

Teachers perceive that their Ed-Tech grant programs have increased student engagement and 
motivation. Most teachers reported that their Ed-Tech grant programs made a noticeable or very 
strong impact on student engagement, with CBTG teachers reporting the strongest impact. In the 
interviews, teachers noted several aspects of the program that promoted engagement and 
motivation: hands-on use of technology, project-based learning, and the opportunity to connect 
with outside experts and peers at other locations. 

Teachers reported that their Ed-Tech programs had an impact on content understanding but not 
on test performance. Over three fifths of the teachers indicated that their Ed-Tech programs have 
increased students’ content understanding; this finding was particularly strong in the CBTG 
Program. This finding may reflect the program’s targeting of specific grade-level expectations. 

In the interviews, teachers attributed this perceived increase in content understanding to greater 
student engagement in tasks and more personalized learning (e.g., self-pacing and additional 
media for learning). Most teachers did not believe that the programs made a noticeable impact on 
student perfonnance on standardized tests. 

Question 5: To What Extent Are Changes in Teaching and Learning Adopted and 
Sustained? 

The majority of the Ed-Tech grant programs have plans for sustainability. Grant managers 
indicated that the majority of the Ed-Tech grant programs will be sustained. In most cases, 
additional local funding will be available to sustain the grant for the purchase of equipment, 
software, and additional or ongoing training. In some instances, such as with the LNV program, 
supplemental funding sources have not been secured, and the sustainability of the grant programs 
at sites without these additional funds is unclear. Across programs, grant managers reported that 
they plan to expand their grants to other teachers in the school and, in some cases, to teachers in 
other schools. In general, teachers who participated in the grant programs have gained the 
knowledge and skills necessary to help train other teachers on what they have learned. 

Question 6: To What Extent Did the Program Support the Development and the 
Deployment of Online Learning Opportunities? 

The Ed-Tech Program funded the creation of the VTVLC program, which had 143 enrollments 
in 17 member schools during its first year. The strongest impact that principals noted from this 
program was on expanding the availability of courses, which is particularly important for the 
rural schools that are common in Vermont. Most principals were satisfied overall with the 
program, and most anticipated participating again in the following year. The courses did not 
strongly incorporate student discussion as a learning tool, even though discussion forums are 
available in all the courses. 

Recommendations 

Recommendation 1: Encourage effective professional development. Effective professional 
development was critical for the success of these grants. It is recommended that VTDOE 
continue to require a percentage of grant funds be devoted to professional development, and this 
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professional development should focus strongly on standards-aligned integration of technology 
into instructional practice. 

Recommendation 2: Provide opportunities for teacher collaboration. The findings from this 
evaluation underscore the importance of teacher collaboration. They suggest that administrators 
of technology integration programs should provide opportunities for teachers to collaboratively 
develop lessons, share resources, and troubleshoot. Administrators should set teacher 
collaboration as an expectation of the grant and ensure that teachers have sufficient opportunity 
for it. 

Recommendation 3: Encourage administrators to actively support implementation of 
technology grants. VTDOE should encourage administrative support for technology grants, 
such as by requiring principals to participate in the project team, stipulating that grantees address 
a current school improvement goal, and have grantees identify in advance their sources of 
technical support. 

Recommendation 4: Consider expanding the programs. In light of the impact on teaching 
practice and student learning, VTDOE should consider expanding these programs. This may 
involve plans for scaling up the grants at other schools so that the success of these programs can 
be shared and replicated. 

Recommendation 5: Encourage grantees to plan for sustainability. To ensure that the work 
of the initial grant is sustained, VTDOE should require grantees to formally articulate their 
sustainability plans. These plans should address the securing of additional funds (if necessary) 
and the expansion of teacher participation. VTDOE may also consider requiring grantees to 
articulate a strategy for parent and community outreach to gamer this support. 

Recommendation 6. Promote student interactions through use of discussion forums. Student 
discussion is a hallmark of quality online teaching (International Association for K-12 Online 
Learning, 2010). To promote the use of discussion forums, VTVLC program leaders should 
consider emphasizing the importance of student interaction in their teacher preparation course 
and monitoring courses to ensure that student interaction is occurring. 
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I. Introduction 



This report presents an evaluation of the Enhancing Education Through Technology (Ed-Tech) 
Program in Vermont, with a focus on the state’s competitive grants. The Ed-Tech Program is a 
funding source authorized under Title II, Part D, of the No Child Left Behind (NCLB) Act of 
2002. In 2009, Ed-Tech funds allocated to Vermont were supplemented by funds from the 
American Reinvestment and Recovery Act (ARRA). The Ed-Tech Program is administered by 
the U.S. Department of Education through its Office of School Support and Technology 
Programs. According to a review conducted by the State Educational Technology Directors 
Association (2010), funding available under the Ed-Tech Program supports NCLB goals in the 
following ways: 

• By closing the achievement gap by providing access to smart computing devices, digital 
content, and open education resources for all students via 21st century learning 
environments that enhance teaching and learning with technology integration 

• By supporting the development of highly qualified teachers with online courses, 
communities of practice, instructional technology coaches, and virtual communication to 
ensure flexibility and access 

• By using data for school improvement and individualizing instruction for all students 

Under these broad parameters, each state education agency (SEA) has the latitude to set its own 
priorities. Each SEA receives its allotment of Ed-Tech funding on the basis of its share of Title I, 
Part A, funding (i.e., funding for economically disadvantaged students). Program guidelines 
stipulate that at least 50 percent — and up to 100 percent — of the available funds must be 
allocated to local education agencies (LEAs) through a competitive grant process. In Vermont, 
one half of the Ed-Tech funds are allocated by formula, which means that districts receive 
funding on the basis of their proportional share of Title I funding. The other half of Ed-Tech 
funds is allocated through competitive grants. These grant programs are designed to support the 
specific goals of the Vermont Ed-Tech Program, which, according to Vermont’s statewide Ed- 
Tech plan (Vermont Department of Education [VTDOE], 2009), are as follows: 

• Lead to changes in classroom practice as teachers participate in professional development 
to learn how to integrate technology into classroom instruction. 

• Increase the ability of principals and other school leaders to support and evaluate teacher 
practices in technology integration through the professional development program 
provided as part of specific programs. 

• Expand student access to flexible learning environments. 

• Increase students’ mastery of the 2 1st century skills 1 required for success in meeting the 
Vennont grade-level expectations for each subject area or discipline by providing 
electronic learning resources for students that are supported by the teacher and school 
leaders’ professional development programs. 



1 Twenty-first century skills are commonly understood to include learning and innovation skills such as critical 
thinking, problem solving, communication and collaboration (see Partnership for 21st Century Skills, n.d.). 
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• Be sustainable and expandable to additional educators beyond the grant years as a result 
of the professional learning networks that are created during the grant program. 

Competitive grants differ from formula grants in two ways. First, prospective grantees are 
evaluated via a competitive grant proposal process. Second, each competitive grant is conferred 
under the guidelines and purposes of a specific competitive grant program. In fiscal year (FY) 
2011, there were eight active competitive grant programs. These competitive grant programs 
differed in structure. Some programs had multiple grantees (each corresponding to a different 
school district or school), whereas others had a single grantee that provided services to numerous 
schools or school districts. Some activities had a statewide focus, providing opportunities for all 
educators and schools in Vermont to benefit from the work within the program. These programs 
serve high-need school districts with activities, professional development, and equipment in line 
with the stated goals of the overall Ed-Tech Program. These activities are driven by Vermont’s 
State Technology Plan (VTDOE, 2009) and were designed to reflect ongoing statewide 
education initiatives. 

Evaluation Purpose and Questions 

American Institutes for Research (AIR) conducted an external evaluation of Vermont’ s Ed-Tech 
Program from January 2010 - June 2011. The purpose of the evaluation was to provide feedback 
on the implementation and outcomes of the competitive grant programs that were active during 
this time. The evaluation did not address fonnula grants. The findings of the evaluation are 
intended to help VTDOE learn from successful components of grant initiatives — so these 
components can be continued or scaled up — and about less successful components — so these 
activities can be improved or funds redirected. This third and final evaluation report builds upon 
two previous reports, submitted in September 2010 and March 2011. This report addresses the 
competitive grant programs that continued or commenced during the 2010-2011 school year. 

During the discovery stage of the evaluation, AIR’s evaluation team worked with VTDOE and 
the coordinators of the different competitive grant programs to specify a set of evaluation 
questions. This process encompassed the following infonnation-gathering efforts: a kickoff 
meeting in Montpelier, Vermont, attended by the evaluation team, the state director of the Ed- 
Tech Program, and the managers of the individual grant programs; the development of program 
profiles based on reviews of the grant documents; and follow-up interviews with the grant 
directors to resolve ambiguities present in the profiles. On the basis of this infonnation, and in 
consultation with VTDOE, the evaluation addressed the following questions: 

7. To what extent and with what fidelity are the grantees of the Ed-Tech Program making 
progress toward their stated objectives? What has facilitated or hindered progress? 

la. To what extent have grant funds been spent so far and on what? 

lb. What technology resources have been and will be purchased, distributed, and 
supported? 

8. How effectively do schools support the implementation of project goals? 

2a. What types of training and professional development were offered? 

2b. What is the scope of participation in professional development? 
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2c. What are participants’ opinions of the quality and the effectiveness of the 
professional development? 

2d. To what extent are teachers provided opportunities to collaborate on implementing 
program objectives for technology integration? 

2e. To what extent do administrators support, advocate, and encourage technology 
integration? 

2f. To what extent and from what sources do teachers receive technology support? 

9. Do the Ed-Tech-funded competitive grant programs promote technology integration in 
support of student-centered learning? 

10. What are the learning outcomes of the program in terms of student engagement and 
motivation and mastery of Vermont grade-level expectations? 

1 1. To what extent are changes in teaching and learning practices adopted and sustained, as 
indicated by continued and expanded use of such practices by teachers and school leaders 
who took part in the program? 

12. To what extent did the program support the development and the deployment of online 
learning opportunities? 

The evaluation was designed to draw conclusions about these questions overall (across the 
different competitive grant programs) and in particular (focusing on findings for particular 
programs). To this end, the evaluation team collected data from both quantitative and qualitative 
data sources (surveys, extant data on participation, interviews, and document review) and 
analyzed these data sources descriptively. The following six competitive grant programs were 
included in the evaluation; these programs are described in detail in Section II: 

• The Content-Based Technology Grant (CBTG) Program 

• The eLearning Project 

• The Vennont Virtual Learning Cooperative (VTLVC) 

• The Learning Network of Vermont (LNV) 

• The Educate/Innovate Program 

• The Real World Design Challenge (RWDC) 

It should be noted that VTDOE contracted with two additional evaluators to conduct external 
evaluations of two individual programs, namely, the 21st Century Classrooms: Connecting the 
Dots Program (DOTS) and the Broadband Consortium Program. 

Report Structure 

The report is organized as follows: 

• Section II. Grant Activities. This section describes the competitive grant activities that 
took place during FY 2011, which are the subject of the evaluation. 

• Section III. Evaluation Design and Methods. 
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• Section IV. The Results. This section summarizes the findings and the conclusions of 
the evaluation conducted by AIR of the competitive grant programs; it is organized by 
evaluation question. Each subsection includes findings that cut across all programs, as 
well as findings specific to the individual programs. 

• Section V. eLearning Spotlight Study. The purpose of this Spotlight Study is to depict 
the fidelity of the implementation of the eLearning Project in relation to its overall project 
plan. This study was undertaken in light of the large investment of the VTDOE in the 
eLearning Project. Moreover, this study was feasible because the eLearning Project is 
comprised of a single grant with a single plan for implementation (that is interpreted and 
applied in the context of six different sites). 

• Section IV. Conclusions and Recommendations. Based on the results of the evaluation, 
this section provides recommendations for expanding and sustaining Vermont’s Ed-Tech 
Program. 
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II. Grant Activities 



This section provides specific details about each competitive grant program included in the 
evaluation, including each program’s goals and expected results. To address the specific goals of 
the Vermont Ed-Tech Program, several competitive grant programs were launched and funded 
by their Title IID and ARRA allocations. The following six programs are described in terms of 
their major goals, the content areas they address (when applicable), the technology provided 
under the grant, the strategies for achieving the goal of the grant, and the intended program 
outcomes. Additionally, each subsection provides information on the program’s implementation 
timeline and level of funding. 

Content-Based Technology Grant Program 

Content-Based Technology Grants (CBTG) were awarded to 53 grantees within 39 schools with 
the goal of providing modest financial assistance to schools with high poverty and the greatest 
need for technology support, including schools identified for improvement under NCLB. 

Original grant awards in FY 2010 ranged from $2,000 up to $10,000 per site. Program 
implementation for the CBTG Program began on January 1, 2010, and ran through March 30, 
2011. Grantees were required to use technology to improve student achievement in at least one 
grade-level expectation related to reading, particularly in the analysis and the interpretation of 
text and the application of that analysis/interpretation to a 21st century literacy skill. Each of the 
individual grants addressed a different Vermont grade-level expectation under the following 
content areas: mathematics and science, art and music, literacy, health, physical education, social 
studies, and world languages. Under the grants, the technology involved varied by individual site 
but could include equipment, hardware, software, books, materials, or other technology 
maintenance supplies needed to fulfill the goal of each grant. 

As a result of the CBTG Program, sites were expected to see changes in teacher practice and 
student learning, as well as student engagement and motivation. Teacher outcomes included 
learning how to use a specific technology and incorporating it into their curricula. In addition, 
teachers were also expected to become leaders within their school teams and school districts, 
demonstrating and leading workshops on the use of technology in the classroom. Furthermore, 
increased student motivation, engagement, interest levels in subject matter, and confidence to 
perfonn tasks and improvements in student scores on local assessment and New England 
Common Assessment Program (NECAP) are expected outcomes of the program. 

Strategies for achieving the grant goals varied by grantee; through the grants, schools developed 
individual programs to integrate technology and equipment into schools based on specific 
content areas. Grantees were also required to use a minimum of 25 percent of the grant funds for 
ongoing, sustained, intensive, high-quality professional development in integrating and using 
advanced technologies in instruction and new learning environments. The most common type of 
professional development was one-to-one or small-group coaching, facilitated by district and 
school technology integration specialists. Other types of professional development included in- 
service training, self-study, workshops, classroom observations, and graduate courses. 
Professional development helped teachers leam how to use the acquired equipment and integrate 
it into their curricula and classroom instruction. 
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In addition to school-based support, CBTG grantees often relied on partners, such as other 
schools, colleges, and educators; nonprofit organizations; museums; professionals and experts in 
the field; and the director of Vermont’s online arts mentoring programs, MIDI and ARTT. These 
partners provided professional development, expertise, technical support, in-kind contributions 
of material resources, and matching funds to achieve the goals of the grant. 

Educate/Innovate Program 

The Educate/Innovate Program offered grants to 17 schools or supervisory unions to incorporate 
various types of technology into teaching and learning in the classroom, in accordance with 
Vennont and national education technology standards. The size of the grants ranged from 
$10,000 to $20,000. Program implementation for the Educate/Innovate Program began on 
October 1, 2010, and ran through June 30, 201 1. As a result of the Educate/Innovate Program, 
sites were expected to develop models of practice that ultimately improve student learning 
outcomes in the core content areas and provide teachers with the skills to use technology and 
incorporate it into their curricula. 

The Educate/Innovate Program operated at two levels: (1) classroom, school, or supervisory 
union focused grants and (2) state-based focused grants. Grants at the first level were either 
school based, with a focus on teachers in that school, or led by a supervisory union or school 
district, where grant resources were spread across a few teachers selected from multiple schools. 
The funds from the grants were intended to support professional development on integrating 
technology into specific content areas. A portion of the grant funds was also directed toward the 
purchase of technology, including hardware and/or software. The goal of these focused grants 
was to use Educate/Innovate funds to develop model classroom practices that can be shared and 
disseminated to the Vermont education community. 

Grants operating with a state-based focus used funds to develop innovative programs and 
professional development opportunities that benefit educators across the state on topics including 
but not limited to the integration of 2 1 st century tools for learning, model policy development in 
areas of need for educators in the use of technology, and best practices in classrooms with social 
networking tools. Under the grants, the technology involved varied by individual site but 
included such things as equipment, hardware, software, books, materials, or other technology 
maintenance supplies needed to fulfill the goal of each grant. 

Grantees used different strategies for achieving the grant goals, including emphasizing teacher 
collaboration, curriculum development, professional development, and the purchase of 
equipment or software. Examples of professional development from across the grant sites 
included on-site sessions with consultants; in-school trainings; workshops; and on-demand, 
online, or Web-delivered professional development. 

Vermont Virtual Learning Collaborative 

The Vennont Virtual Learning Collaborative (VTVLC) provides K-12 programs and courses in 
a wide variety of subject areas by partnering with schools to offer online classes to students 
around the state. Seven career and technical education regions were selected to participate in 
phase 1 of the VTVLC program, representing more than 30 high schools and middle schools. In 
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phase 2 of the program, occurring in FY 201 1, 32 teachers were trained from 17 schools. In FY 

201 1, a single grant of approximately $387,548 was awarded to the River Valley Technical 
Center to coordinate these activities. Of the funds awarded, $242,274 were from holdover FY 
2010 ARRA funds and $145,274 were from the FY 2011 allocation of regular Title IID funds. 
VTVLC program implementation began on January 15, 2010, and will run through June 30, 

2012. VTVLC is anticipated to continue to be developed as it required a sustainability plan that 
is currently on track and succeeding. 

VTVLC is intended to benefit students by providing opportunities for learning that extend 
beyond the classroom day. The grant program is also aimed at meeting the instructional 
improvement and student achievement goals of the Ed-Tech Program by offering professional 
development for teachers, guidance counselors, and administrators on topics that involve online 
education and flexible learning opportunities for students. Through the VTVLC grant, 
participating schools do not receive technology in terms of equipment, hardware, or software. 

The VTVLC program launched during the summer in 2010 across 14 schools, with 75 students 
enrolled in full-credit courses. In fall 2010, 143 students across 19 schools enrolled in VTVLC 
online courses, which were expanded to include art and technology courses. The program has 
expanded to 3 1 schools, and a total of 32 online teachers were trained in the 2010-11 academic 
year. 

Learning Network of Vermont 

The Learning Network of Vermont (LNV) is a videoconferencing network, currently operating 
in 104 elementary, middle, and high schools in Vermont. There was a single grant of 
approximately $222,354 made in FY 2011 from holdover Regular IID funds from FY 2009 and 
FY 2010. The grant was awarded to the Essex Caledonia Supervisory Union to coordinate these 
activities. LNV program implementation began on October 1, 2008, and ran through June 30, 
2011 . 

LNV was intended to benefit students by providing opportunities for learning that extend beyond 
the classroom walls, including collaborations with other classrooms and students, virtual field 
trips, and visits by experts in various professional fields in real time. Teachers are also to benefit 
from using the system to acquire skills through professional development offerings that mitigate 
geographic barriers for collegiality and professional meetings. 

LNV was created with the goal of promoting flexible learning environments for students. 
Through the Ed-Tech grant, LNV was able to increase the system’s overall capacity, as well as 
fund access to the Center for Interactive Learning and Collaboration (CILC) content bank with 
over 900 possible videoconferencing sessions from a wide variety of providers. Grant funds 
allowed LNV to create a website of resources and information, develop an online scheduling and 
meeting control system, provide professional development, and offer a mini-grant program to 
participating sites offering cameras. 
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eLearning Project 



The eLearning Project was a program that provided professional development services to six 
demonstration sites to serve as models for other schools making the transition to 2 1 st century 
learning in Vermont. Although the grant was initially funded through December 2010, it was 
extended through May 2011. A single grant of approximately $55,000 was made in FY 2011 
from holdover FY 2010 ARRA funds. The grant was awarded to the Tarrant Center at the 
University of Vermont and the Milton School District to coordinate this project. Program 
implementation for the eLearning Project began on September 1, 2009, and ran through June 30, 
2011 . 

The eLearning Project was intended to benefit schools across Vermont by establishing model 
regional sites that would provide professional development and best practices in the 
implementation of technology-rich classroom activities. The grant program was also aimed at 
meeting the instructional improvement and student achievement goals of Ed-Tech Program, as 
highlighted by the regional component that reaches high-need and low-achievement schools in 
all corners of the state. This program also provided professional development that changed 
teacher practice in significant ways with the integration of technology. 

The purpose of the eLearning Project was to assist teachers and school leaders, through research- 
based professional development, to become more proficient in effective student-centered, 
technology-rich teaching and learning. Through the eLearning Project, participants also had 
access to an interactive network of resources that supported their ongoing work, with the 
ultimate goal of establishing a statewide communication network to support educators in their 
work. 

At each site, there was a core project team consisting of five to seven teachers, the school 
principal, and a technology specialist. This team worked with an external coach (paid for and 
assigned by the overall project leadership) to accomplish a specific scope of work. Each project 
site developed its own scope of work, which was referred to as a syllabus. Each project team was 
expected to develop resources for technology integration and to share these resources via an 
interactive website. Each site was also expected to plan and host three regional workshops open 
to teachers in their own school and neighboring schools. Technology was not distributed at the 
school level as part of this grant program. 

Real World Design Challenge 

Real World Design Challenge (RWDC) is a national competition that is sponsored and funded by 
each state that chooses to participate. The program began in 2008 with a partnership between the 
U.S. Department of Energy and Parametric Technology Corporation in Massachusetts. During 
the pilot year of the program, 10 states were involved, including Vermont. Program 
implementation for RWDC in Vennont began on September 1, 2010, and ran through June 30, 
2011 . 

The Vennont RWDC Ed-Tech grant was intended to benefit schools by providing them with 
support to make them viable RWDC candidates from which to choose a national contestant. The 
focus of the Vermont RWDC Ed-Tech grant was to select and fund an entity to manage and lead 
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the 2010-11 version of RWDC, including recruiting schools to take part in the challenge; 
assisting in the coordination of summer and fall trainings to schools; visiting and supporting 
those schools taking part in the challenge; and representing Vennont in the continued work the 
challenge represents. 

Through the program, student participants were expected to gain skills in math, science, and 
computer-aided drafting programs. Schools participating in the RWDC program received 
software and training to launch the program at their schools. Students also received software to 
use on home computers. The intent of the program was to rebuild the ranks of students 
throughout the United States that are learning science, technology, mathematics, and engineering 
skills (STEM). The purchase of technology was not part of this grant program, though some 
limited hardware purchases are permitted for the purposes of staff and coordinator work duties. 
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III. Evaluation Design and Methods 



The evaluation team collected data using surveys of teachers and grant managers and interviews 
of school and district staff. Table 1 describes the alignment of these data sources to the 
evaluation questions. 



Table 1. Alignment of Data Sources to the Evaluation Questions 



Evaluation Question (Abbreviated) 


Teacher 

Survey 


Grant Manager 
Survey 


Site 

Visits 


1 . Progress toward objectives 




X 




1 a Spending of grant funds 




X 




lb. Technology resources purchased 




X 




2. Support for implementation 


X 


X 


X 


2a. Types of professional development 


X 


X 


X 


2b. Scope of participation in professional development 




X 


X 


2c. Opinions of professional development 


X 




X 


2d. Opportunities to collaborate 


X 




X 


2e. Administrative support 


X 




X 


2f. Technology support 


X 




X 


3. Changes in teacher practice 


X 




X 


4. Student learning outcomes 


X 




X 


5. Sustainability 




X 


X 


6. Online learning opportunities 3 






X 



a This question is also addressed with a principal survey and extant data on program participation. 



For the VTVLC program, there were two additional data sources: a principal survey and extant 
data describing student participation and grades in online courses. These data sources are 
described later in the report in the section addressing Evaluation Question 6. 

Teacher Survey 

Survey Design and Validation. The teacher survey addressed the topics with which teachers 
had direct experience, including attendance at professional development events, the quality and 
the impact of professional development, the frequency and the impact of teacher collaboration, 
administrative direction and support, barriers to implementation, the impact on teacher practice, 
the frequency of student use of technology for different purposes, and the impact of the program 
on student engagement and learning. Some of these topics were considered to be constructs, 
namely, a core idea that is measured by a series of survey items. We used the Rasch model for 
ordered categories to determine whether the items within each construct could be combined into 
a single scale score (Andrich, 1978; Wright & Masters, 1982). The following constructs were 
identified through this approach: 

• Impact on teacher practice — instruction. The items in this construct asked teachers to 
rate the program’s impact on instructional practices related to technology integration, 
such as project-based learning, the use of digital tools, active learning, and student 
collaboration with technology. The four point rating scale was as follows: no 
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impact/negative impact, slight impact (to date), noticeable impact, and very strong 
impact. The items in this construct are informed by the International Society for 
Technology in Education’s (ISTE) National Educational Technology Standards for 
Teachers (NETS-T; ISTE, 2008), particularly the standards “Design and develop digital- 
age learning experiences and assessments” and “Facilitate and inspire student learning 
and creativity.” The individual items comprising the Impact on Instructional Practice 
scale are as follows: 

■ Incorporating digital tools and resources into lessons 

■ Providing opportunities for students to engage in interactive learning (in contrast to 
lecture-based instruction) 

■ Providing opportunities for students to work on extended projects (a week or more in 
duration) 

■ Providing opportunities for students to collaborate 

■ Customizing learning activities to address students’ diverse ability levels 

■ Providing opportunities for students to explore real-world problems 

• Impact on teacher practice — assessment. The items in this construct asked teachers to 
rate, using the same scale as given previously, the program’s impact on assessment 
practices, such as designing assessments that are appropriate for project-based learning. 
These items are drawn from the ISTE NETS-T standard, “Design and develop digital-age 
learning experiences and assessments.” The individual items comprising the Impact on 
Assessment Practices scale are as follows: 

■ Using student assessments to inform future lessons 

■ Designing assessments that are appropriate for project-based learning 

■ Collecting various kinds of evidence of student learning 

■ Using student assessments to differentiate instruction 

Based on the survey scaling process for each construct, a respondent could be categorized by his 
or her typical response (e.g., no impact/negative impact, slight impact (to date), noticeable 
impact, and very strong impact). 

A third construct, professional development quality, was also evaluated, but it could not be 
reliably combined into a single scale score. All other items on the survey were designed to be 
analyzed and reported individually. A copy of the teacher survey is included in Appendix A. 

Administration Procedures and Response Rate. The teacher survey was administered online, 
with individual invitations sent to 266 teachers who were affiliated with 5 of the 6 competitive 
grant programs; teachers participating in VTVLC were not invited to participate in the survey 
because the topics covered by the survey did not relate to the purpose of the VTVLC program. 
Teachers were given 30 days to complete the survey and were sent 4 e-mail reminders during 
this window. A total of 165 of 266 teachers associated with 5 programs responded to the online 
survey (for a 62 percent response rate). Table 2 provides the response rate for each program and 
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across programs." The eLearning Project had the highest response rate, with 95 percent of its 
teachers responding to the survey. 



Table 2. Respondents by Grant Program (N = 165) 



Grant Program 


Number of 
Invitations Sent 


Number of 
Respondents 


Response Rate 
(Within Program) 


Percentage of Total 
Respondents (Across 
All Programs) 


CBTG 


127 


73 


57.5% 


44.2% 


Educate/Innovate 


72 


48 


66.7% 


29.1% 


eLearning Project 


20 


19 


95.0% 


11.5% 


LNV 


42 


18 


42.9% 


10.9% 


RWDC 


5 


3 


60.0% 


1.8% 


Other 


— 


4 


— 


2.4% 


Total 


266 


165 


62.0% 


100.0% 



Note. The program affiliation of four respondents could not be determined. 



Survey Analysis. The response frequencies were examined in the aggregate and by program. 
Where response frequencies differ notably by program, the survey findings are presented for 
each program. Where there is little variation by program, the response frequencies are presented 
in aggregate. However, for those items, a breakout of findings by program is provided in 
Appendix B. Due to the fact that only three RWDC teachers responded to this survey, the 
findings of these respondents are not broken out by program. They are included in the aggregated 
survey results. By the same token, the responses of four teachers whose program affiliation could 
not be detennined are also included in the aggregated survey results. 

Grant Manager Survey 

The grant manager survey addressed the evaluation questions grant managers were most 
knowledgeable about, which included the use of grant funds and program participation. For the 
CBTG Program and the Educate/Innovate Program, grant managers are typically a school or a 
supervisory union staff member — including teachers, principals, and technology coordinators — 
who are responsible for overseeing the implementation of the grant. The eLearning Project, 

LNV, and RWDC each have one grant manager at the state-level that is tasked with overseeing 
funding for each individual grant, as well as coordinating professional development events and 
ensuring that the programs are implemented statewide. The VTVLC grant manager was not sent 
this survey because the items addressed technology integration programs, rather than online 
learning efforts. A copy of the grant manager survey is included in Appendix A. 

Administration Procedures and Response Rate. The grant manager survey was administered 
online, with individual invitations sent to 63 grant managers. Grant managers were given 30 days 
to complete the survey and were sent four e-mail reminders during this window. Forty grant 
managers responded to the survey (for a 64 percent response rate). As shown in Table 3, the 



2 There was no basis other than program affiliation to distinguish respondents from non-respondents on this survey 
or the grant manager survey. 
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majority of respondents (62 percent) indicated that they are participants in the CBTG Program. 
As noted, the eLearning Project, LNV, and RWDC have one grant manager each. 



Table 3. Respondents by Grant Program (N = 39) 



Grant Program 


Number of 
Invitations Sent 


Number of 
Respondents 


Response Rate 
(Within 
Programs) 


Percentage of Total 
Respondents (Across 
All Programs) 


CBTG 


44 


25 


56.8% 


62.5% 


Educate Innovate 


16 


11 


68.8% 


27.5% 


eLearning Project 


1 


1 


100.0% 


2.5% 


LNV 


1 


1 


100.0% 


2.5% 


RWDC 


1 


1 


100.0% 


2.5% 


Other 


— 


1 


— 


2.5% 


Total 


63 


40 


63.5% 


100.0% 



Note. The program affiliation of one respondent could not be determined. 



Interview Methodology 

The interview methodology involved selecting up to four sites for each program and conducting 
a series of interviews at each site. Typically, the interviews were conducted with participating 
teachers, the grant manager, and the school principal. For some programs, additional types of 
interviews were conducted, as explained here. Interviews were generally conducted on-site at the 
participating schools. In fall 2010, four site visits were conducted at (each) of the CBTG, 
VTVLC, and eLearning Project sites, and 5 site visits were conducted at LNV sites, for a total of 
17 sites. During spring 2011, the evaluation team conducted eight additional site visits at four 
Educate/Innovate and four eLearning sites. The number of site visits conducted per program 
during each semester is summarized in Table 4. 



Table 4. Number of Site Visits Conducted by Program and Semester 



Grant Program 


Number of Fall 2010 Site Visits 


Number of Spring 2011 Site Visits 


CBTG 


4 


0 


Educate/Innovate 


0 


4 


eLearning Project 


4 


4 


LNV 


5 


0 


VTVLC 


4 


0 


RWDC 


0 


0 



Fall 2010 Site Visits. Across most programs represented in the fall 2010 site visits, schools were 
selected to maximize diversity across the geographic region. However, VTVLC sites were 
selected to maximize diversity in terms of the intended purpose of the school’s participation 



3 Additional data collection was not conducted at the LNV, CBTG, and VTVLC sites in spring 2011 because their 
grantees had, for the most part, completed their implementation goals for the grant. The Education/Innovate 
Program had been at too early a stage of implementation during fall 2010 for site visits to be conducted then. The 
four eLearning Project sites were selected for an additional round of site visits that focused on program fidelity and 
sustainability; these questions could not be adequately addressed at each site’s stage of implementation in fall 2010. 
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(e.g., credit recovery versus enrichment) and urbanicity. For most programs, site visits involved 
interviews with multiple individuals. One exception was the LNV program, where the interviews 
were conducted by telephone and typically involved one teacher per school, as participation in 
this program was usually limited to a small number of individuals per school. For this program, a 
convenience sample was drawn based on the most frequent or recent users of the system, as 
recommended by the program’s administrator. The distribution of respondents by type and 
program is presented in Table 5. 



Table 5, Number of Interviewees by Program, Fall 2010 



Program 


Schools 


Teacher Only 


Teacher and 
Grant Manager 3 


Grant 

Manager Only 3 


Principal 


CBTG 


4 


8 


6 


3 


3 


eLearning Project 


4 


9 


2 


3 


4 


LNV 


5 


6 


N/A 


N/A 


N/A 


VTVLC b 


4 


6 


N/A 


N/A 


4 


Total 


17 


29 


8 


6 


11 



“For the eLearning Project, the site’s team leader was interviewed as the grant manager. b For VTVLC, 6 interviews with guidance 
counselors and 6 student focus groups were also conducted. 



For the CBTG Program, the eLearning Project, and LNV, a single set of interview protocols was 
designed to align with the evaluation questions. To understand student experiences in the 
program, we also conducted student focus groups. The majority of the interviews were 
conducted on-site at the interviewee’s school, although five interviews (three for LNV, one for 
the eLearning Project, and one for the CBTG Program) were conducted by telephone. Each 
interview lasted between 45 and 60 minutes. Interviewers took notes during the interviews, and 
all the interviews were audio recorded, with the participant’s consent, and transcribed for 
analysis. Because of programmatic differences, a unique set of protocols was developed for the 
interviewees at the VTVLC sites. Unlike the other programs, at the VTVLC sites we conducted 
interviews with guidance counselors (who typically manage student registration). 

Spring 2011 Site Visits. During spring 2011, site visits were conducted at four Educate/Innovate 
and four eLearning Project schools. The Educate/Innovate sites were sampled to maximize 
diversity in terms of the grade span addressed by the program and the program’s structure (i.e., 
whether the grantee was comprised of a single school district or supervisory union). For those 
grantees with multiple sites, the grant coordinator helped identify a single school from which to 
interview staff. 

Thirteen interviews were conducted with individuals from the Educate/Innovate Program. 
Interviews were approximately 45 minutes in duration and conducted with one or two teachers, 
the school principal, and the grant coordinator. Eight interviews for this program were conducted 
in person, and 5 were conducted by phone. 

Interviews with eLearning Project participants were conducted at four sites. 4 Thirteen interviews 
were conducted across eLearning Project coaches, teachers, and the school principal. The 

4 Three of these sites had been visited in fall 2010. One site was replaced after consultation with the eLearning 
Project director because it was being implemented in a setting that was highly unique and not representative of most 
schools. 
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interviews with coaches were 45 minutes in duration. Shorter follow-up interviews ranging from 
15 to 20 minutes in duration were conducted with teachers and principals (several of whom were 
interviewed in fall 2010). Ten eLearning Project interviews were conducted by phone, and one 
was conducted in person. Due to the brevity of the interview questions, teachers and principals 
from eLearning Project schools were also invited to respond to the questions in writing. Two 
teachers submitted responses in writing. See Table 6. 



Table 6. Number of Interviewees by Program, Spring 2011 



Program 


Schools 


Teacher 


Grant Manager 


Coach 


Principal 


Educate/Innovate 


4 


7 


4 


N/A 


3 


eLearning Project 


4 


5 


N/A 


4 


4 


Total 


8 


12 


4 


4 


7 



Interview Analysis. The analysis of qualitative data employed an inductive approach that 
incorporated systematic methods of managing data through reduction, organization, and 
connection (Dey, 1993; LeCompte, 2000). This process relied on systematic procedures for 
coding and categorizing the data to recognize patterns within and across schools. The evaluation 
team inductively analyzed interview transcripts by scanning the data for categories of 
phenomena and relationships among such categories. 

The analysis of the interviews employed a two-step process. First, the evaluator developed site 
profiles based on the responses of all individuals from each school. These profiles were 
organized according to the topics included in the evaluation questions. Second, a cross-site 
analysis was conducted for each program to identify themes across school profiles. 
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IV. The Results 



Question 1: To What Extent and With What Fidelity Are the Grantees 
Making Progress Toward Their Objectives? 

This subsection summarizes the findings aligned to the overall evaluation question, as well as to 
the following subquestions: 

la. To what extent have grant funds been spent so far and on what? 

lb. What technology resources have been and will be purchased, distributed, and supported? 

This subsection begins with a summary of the prevalent content areas, goals, and activities as 
indicated by the grant managers. 

Topic Areas of Focus 

As shown in Table 7, grant managers across programs most frequently indicated that technology 
integration (38 percent), English language arts (32 percent) and science (25 percent) were major 
content areas of focus for their programs. Grant managers for the CBTG Program and the 
Educate/Innovate Program reported a variety of content areas, consistent with the structure of 
those programs. Every content area was addressed, typically by several grants. 

Goals of Focus 

Grant managers were also asked to select from a list of goals that represented the major goals of 
their programs. Two of these goals — technology integration and improving achievement in 
content areas — aligned with the statewide Ed-Tech goal of integrating technology in classroom 
instruction. Two other goals — promoting 21st century skills and increasing project-based 
learning — aligned with the statewide Ed-Tech goal of promoting 21st century skills. The most 
frequently selected major goals were technology integration (82 percent) and promote 21st 
century skills (79 percent). However, the remaining three goals were each selected by at least 
55 percent of the grant managers. These results are summarized by program in Table 8. Thus, the 
grant managers indicated that the goals of their grants related to the overall goals of the statewide 
program. 
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Table 7. Content Areas of Focus as Indicated by Grant Managers (N = 40; select all that apply) 



Which of the following content areas was a major 
focus of your program? (check all that apply) 


Overall 
(A = 40) 


CBTG 
(n = 25) 


Educate/ 
Innovate 
(n = 11) 


eLearning 
Project 
(» = 1) 


LNV 
(n = 1) 


RWDC 
(n = 1) 


VTVLC 
(» = 1) 


Technology integration 


37 . 5 % 


24.0% 


45.5% 


X 


X 


X 


X 


English language arts 


32 . 5 % 


28.0% 


36.4% 




X 




X 


Science 


25 . 0 % 


20.0% 


18.2% 




X 


X 


X 


Social studies 


20 . 0 % 


16.0% 


18.2% 




X 




X 


PE/health 


20 . 0 % 


24.0% 


0.0% 




X 




X 


Arts 


17 . 5 % 


20.0% 


9.1% 










Technology literacy 


17 . 5 % 


4.0% 


36.4% 






X 


X 


Technology education (ITEA Standards) 


15 . 0 % 


8.0% 


27.3% 






X 




Mathematics 


12 . 5 % 


4.0% 


27.3% 








X 


World languages 


12 . 5 % 


8.0% 


9.1% 




X 




X 


Career and technical education 


2 . 5 % 


0.0% 


0.0% 






X 





Note: ITEA = International Technology Education Association 



Table 8. Proportion of Grant Managers Endorsing Different Program Goals as Major 



To what degree was each of the following a 
major goal of your Ed-Tech grant program? 


Overall 
(A =40) 


CBTG 
(n = 25) 


Educate/ 
Innovate 
(« = 11) 


eLearning 
Project 
(» = 1) 


LNV 
(» = 1) 


RWDC 
(» = 1) 


VTVLC 
(n = 1) 


Technology integration 


82 . 1 % 


80.0% 


80.0% 


X 


X 


X 


X 


Promote 2 1 st century skills 


79 . 5 % 


80.0% 


90.0% 


X 


X 






Improve students’ skills and/or achievement levels 
in a given content area 


67 . 5 % 


76.0% 


63.6% 






X 




Gain resources for the school 


66 . 7 % 


84.0% 


30.0% 




X 




X 


Increase opportunities for project-based learning 


55 . 3 % 


56.0% 


55.6% 






X 
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Across the four grant programs included in the interviews — the eLearning Project, the 
Educate/Innovate Program, the CBTG Program, and LNV — respondents were asked to describe 
the main purpose or motivation for the grant program at their schools. The overall purposes of all 
four grant programs were to (1) promote schoolwide technology integration in the classroom, 

(2) provide teachers with professional development, and (3) enhance student learning using 21st 
century tools and teaching pedagogies. One teacher commented, 

The big picture idea is that we’re trying to transform teaching and learning in ways that 
technology enhances. So we’re moving toward project-based learning, place-based 
learning, [and] more collaboration to teach students 21st century skill sets. 

Although the overall purpose is consistent across the four grants, the grants also have purposes 
intended for their specific programs. eLearning Project schools share the specific purpose of 
becoming model schools for others to visit and learn, while the CBTG and the Educate/Innovate 
programs focus on acquiring equipment for specific content areas. In addition, LNV funding and 
equipment is intended to supplement existing curriculum and promote collaboration and 
communication among teachers and students. Overall, these goals and purposes reflect the ones 
endorsed on the grant manager survey. 

Activities of Focus 

As a part of the survey, grant managers selected which activities were a minor or a major focus 
of their grants. As shown in Table 9, the majority of grant manager survey respondents indicated 
the following activities were a major focus: the purchase of equipment or software (75 percent), 
teacher collaboration (64 percent), professional development (62 percent), and curriculum 
development (52 percent). These foci reflect the cross-program focus on both technology 
integration and professional learning. Grant managers also reported that community outreach 
was not a major focus. 

As indicated in Table 9, program activities described as major varied by program. Professional 
development and teacher collaboration were considered to be the major foci of slightly more than 
one half of the CBTG Program managers, whereas a much larger proportion of grant managers 
from the Educate/Innovate Program (and most of the other grant managers) indicated these were 
major foci. 

Grant managers reported their overall progress with implementation through the grant manager 
survey, which asked respondents to select the stage of program implementation that best 
described their grant projects. The results are summarized in Table 10. Although the programs 
are in various stages of implementation, 72 percent of the grant managers reported that they were 
either in full implementation (22 percent), modification (20 percent), or sustainability 
(30 percent). For the three respondents who reported “other,” two from the Educate/Innovate 
Program reported that more than one pilot project or site is participating under their grant and 
they are in a variety of stages of implementation. For the eLearning Project, the grant manager 
indicated that the grant program’s funding ended on June 30, 2011, and sites have or are in the 
process of making plans to continue their work. 
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Table 9. Proportion of Grant Managers Endorsing Different Program Activities as “Major” 



To what degree does your Ed-Tech grant program focus on 
the following activities? 


Overall 
(A =40) 


CBTG 
(n = 25) 


Educate/ 
Innovate 
i n = 11) 


eLearning 
Project 
(» = 1) 


LNV 
(» = 1) 


RWDC 
in = 1) 


VTVLC 
in = 1) 


Purchase of equipment or software 


75.0% 


100.0% 


45.5% 










Teacher collaboration 


64.1% 


52.0% 


90.0% 


X 


X 


X 




Professional development 


62.5% 


56.0% 


72.7% 


X 


X 




X 


Curriculum development 


52.5% 


44.0% 


81.8% 








X 


Installation of equipment or software 


29.7% 


43.5% 


0.0% 






X 




Parental/community outreach 


12.8% 


8.0% 


20.0% 






X 





Table 10. Current Stage of Program Implementation, as Indicated by Grant Managers (A = 40) 



What is the current stage of program implementation to 
meet the goals of your grant? 


Overall 
(A = 40) 


CBTG 
in = 25) 


Educate/ 

Innovate 

(«=11) 


eLearning 
Project 
in = 1) 


LNV 
in = 1) 


RWDC 
in = 1) 


VTVLC 
in = 1) 


Early implementation — Expectations for the program are high, 
but new skills and appropriate support systems are still fragile 
and developing. 


20.0% 


16.0% 


36.4% 










Full implementation — Program is fully operational. 


22.5% 


36.0% 


0.0% 










Modification — Program is undergoing refinement based on 
feedback or assessment. 


20.0% 


24.0% 


9.1% 




X 






Sustainability — Focus is on the long-term survival and 
continued effectiveness of the program. 


30.0% 


24.0% 


36.4% 






X 


X 


Other (please specify) 


7.5% 


0.0% 


18.2% 


X 
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In contrast, two thirds of the grant managers previously reported that for the 2009-10 school year 
they were either in “program installation” (14 percent) or “early implementation” (51 percent), 
indicating that most of the grants have moved from the early stages of implementation during the 
2009-10 school year to full implementation during the 2010-11 school year. 

la. To What Extent Have Grant Funds Been Spent So Far and on What? 

The grant manager survey asked respondents to report the proportion of their grant funds 
expended to date, and what percentage of the funds allocated to professional development had 
been spent so far. Additionally, the survey asked respondents to anticipate when grant funds 
would be fully expended. All respondents reported that at least 41 percent of their grant funds 
had been expended to date, with the majority (78 percent) indicating that they have spent 
81 percent to 100 percent of their grant funds to date (see Table 1 1). This is consistent with the 
planned funding cycle. 

Most grant managers indicated that they have expended all of their professional development 
funds. As Table 12 shows, the majority of the respondents (68 percent) indicated that between 
81 percent and 100 percent of the funds devoted to professional development have been spent. 
This figure was substantially lower (45 percent) for Educate/Innovate grant managers, whose 
grants started later than other programs. For those grant managers who indicated that they are 
“unsure,” all are managers for grants where funds were divided among multiple sites. 
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Table 11. Amount of Grant Fund Expended to Date as Indicated by Grant Managers (A = 40) 



Approximately what proportion 
of your grant funds have you 
expended to date? 


Overall 
(A =40) 


CBTG 
(n = 25) 


Educate/ 
Innovate 
(n = 11) 


eLearning 
Project 
(» = 1) 


LNV 
(» = 1) 


RWDC 
(» = 1) 


VTVLC 
(» = 1) 


4 1 percent to 60 percent 


7.5% 


4.0% 


18.2% 


X 








61 percent to 80 percent 


7.5% 


8.0% 


0.0% 






X 




8 1 percent to 1 00 percent 


77.5 % 


88.0% 


63.6% 




X 




X 


Unsure 


7.5% 


0.0% 


18.2% 


X 









Table 12. Percentage of the Funds Allocated to Professional Development Spent as Indicated by Grant Managers (A = 40) 



What percentage of the funds 
allocated to professional 
development has been spent? 


Overall 
(A =40) 


CBTG 
(n = 25) 


Educate/ 
Innovate 
(n = 11) 


eLearning 
Project 
(» = 1) 


LNV 
(» = 1) 


RWDC 
in = 1) 


VTVLC 
in = 1) 


0 percent to 20 percent 


7.5% 


8.0% 


9.1% 










2 1 percent to 40 percent 


7.5% 


4.0% 


9.1% 




X 






4 1 percent to 60 percent 


2.5% 


0.0% 


9.1% 










61 percent to 80 percent 


7.5% 


4.0% 


18.2% 










8 1 percent to 1 00 percent 


67.5% 


84.0% 


45.5% 








X 


Unsure 


7.5% 


0.0% 


9.1% 


X 




X 





Note. Not all columns total 100.0% due to rounding. 
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Grant managers were also asked to indicate when they expect their grant funds to be fully 
expended. As Table 13 shows, nearly all the grant managers projected that their grant funds 
would be expended by June 2011, and nearly one half (45 percent) reported that their grant funds 
were fully expended as of March 2011. 



Table 13. Month and Year Grant Funds Are Projected to Be Fully Expended 
as Indicated by Grant Managers (N = 36) 



When do you anticipate grant funding to 
be fully expended? 


Percentage of Total 
Respondents 


Cumulative Percentage 


December 2010 


13.9% 


13.9% 


January 2011 


25.0% 


38.9% 


March 2011 


5.6% 


44.5% 


May 2011 


2.8% 


47.3% 


June 2011 


50.0% 


97.3% 


August 20 1 1 


2.8% 


100.0% 



In summary, grant managers typically reported having spent the majority of grant funds overall, 
as well as those funds allocated to professional development. The few exceptions were grantees 
whose programs began during the 2010-11 school year. 

lb. What Technology Resources Have Been Purchased, Distributed, and Supported? 

As shown in Table 14, grant managers reported variation in the types of technology resources 
that were purchased across the Ed-Tech grants. Nearly all the grant managers reported putting at 
least some grant funds toward the purchase of professional development, with over three fourths 
allocating at least 20 percent of their grant budgets to professional development. Additionally, 
more than one half of the grant managers reported that grant funds were used to purchase laptop 
computers and/or audiovisual equipment. On the other hand, less than 20 percent of the grant 
managers reported spending grant funds for the purchase of interactive whiteboards, virtual- 
learning course content, desktop computers, and/or virtual-learning management systems. 



Table 14. Grant Fund Expenditures by Resource as Indicated by Grant Managers 



Approximately what percentage of 
grant funds has been spent on the 
following? 


N 


Percentage of Budget Allocated 


None 


1 - 

19% 


20- 

39% 


40- 

59% 


60- 

79% 


80- 

99% 


Professional development 


37 


2.7% 


21.6% 


51.4% 


8.1% 


8.1% 


8.1% 


Evaluation 


35 


8.6% 


80.0% 


2.9% 


0.0% 


0.0% 


8.6% 


Laptop computers 


33 


39.4% 


21.2% 


3.0% 


18.2% 


6.1% 


12.1% 


Visual or audio equipment 


33 


48.5% 


24.2% 


15.2% 


3.0% 


3.0% 


6.1% 


Curriculum development 


32 


53.1% 


28.1% 


6.3% 


3.1% 


0.0% 


9.4% 


T echnology integrator/ coach/mentor 


32 


56.3% 


25.0% 


6.3% 


3.1% 


3.1% 


6.3% 


Handheld devices 


37 


62.2% 


16.2% 


2.7% 


2.7% 


10.8% 


5.4% 


Other 


11 


63.6% 


18.2% 


9.1% 


0.0% 


9.1% 


0.0% 
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Approximately what percentage of 
grant funds has been spent on the 
following? 


N 


Percentage of Budget Allocated 


None 


1 - 

19% 


20- 

39% 


40- 

59% 


60- 

79% 


80- 

99% 


Software/courseware 


31 


67.7% 


22.6% 


3.2% 


0.0% 


0.0% 


6.5% 


C onnectivity/ networking 


32 


75.0% 


9.4% 


9.4% 


0.0% 


0.0% 


6.3% 


Interactive whiteboards 


32 


81.3% 


6.3% 


3.1% 


3.1% 


3.1% 


3.1% 


Website development 


30 


83.3% 


13.3% 


0.0% 


0.0% 


0.0% 


3.3% 


Virtual-learning course content 


31 


83.9% 


9.7% 


3.2% 


0.0% 


0.0% 


3.2% 


Desktop computers 


31 


87.1% 


9.7% 


0.0% 


3.2% 


0.0% 


0.0% 


Virtual learning management system 


32 


87.5% 


9.4% 


0.0% 


0.0% 


0.0% 


3.1% 



The interviews with grant managers corroborated these findings. Typical equipment purchased 
for the Educate/Innovate and the CBTG programs included document cameras, netbook 
computers, software, headsets with microphones, projectors, digital flip cameras, smart pens, 
interactive whiteboards and LCD projectors, and iPod Touch units. LNV schools each received a 
Polycom voice conferencing system, a high-definition camera, a speaker unit, a dialer unit, and a 
computer console. The schools paid for or acquired through other resources the high-definition 
televisions for the program to run properly. eLearning Project grantees did not purchase 
technology with grant funds. Some of the technology acquired by grantees was free of cost. 
Participants from the eLearning Project, the Educate/Innovate Program, and the CBTG Program 
reported using Google Apps and other free, online resources for technology integration. LNV 
grantees use their videoconferencing equipment gained through the program as well as the 
credits they received for access to fee-based content provided by CILC. 

Summary 

In summary, the foci of the grant programs were well aligned with the statewide Ed-Tech 
Program goal of integrating technology with classroom instruction. The different grants 
addressed a wide variety of content areas, with English language arts and science being the most 
frequent. The most frequent major goals of each program were technology integration and the 
promotion of 21st century skills. Moreover, the grantees frequently focused on teacher 
collaboration and professional development, reflecting the statewide strategy of professional 
learning. Grant programs seldom focused on parental or community outreach. 

The majority of the Ed-Tech grant programs are currently in the latter stages of implementation, 
with some variation because the Educate/Innovate Program began during the 2010-11 school 
year. Across all programs, over three fourths of the grant managers reported they have expended 
between 8 1 percent and 1 00 percent of the funds to date, including funds dedicated to 
professional development. 

Question 2: How Effectively Do Schools Support the Implementation 
of Project Goals? 

Given that the majority of the grant programs are in the later stages of implementation, this 
subsection focuses on the support provided to grant recipients, including professional 
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development, administrative support for the program, and opportunities for teachers to 
collaborate on integrating technology. 

2a. What Types of Training and Professional Development Were Offered? 

Grant managers indicated which professional development formats had been offered as part of 
their Ed-Tech-funded grant programs. Across all programs, grant manager survey respondents 
indicated that on-site sessions with a consultant (70 percent), in school training (70 percent), and 
workshops (63 percent) were the most common forms of professional development offered as a 
part of their grant programs. These are consistent with the forms of professional development 
that grant managers indicated were offered during the 2009-10 school year. As Table 15 shows, 
the types of professional development events offered varied only slightly by program. 

Consistent with the grant manager survey responses, across all programs, respondents to the 
teacher survey indicated that they most commonly reported attending single day, multihour 
sessions, such as in-service or professional development days, as shown in Table 16. Of those 
who selected “other,” the responses included college courses, presenting at conferences, and 
monthly afterschool meetings. The major difference by program was for coaching from an 
external consultant, which was selected by 32 percent of the teachers overall but by 79 percent of 
the eLearning Project respondents. In addition, relatively few LNV teachers received coaching 
either from an external consultant or a school staff member. 
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Table 15. Professional Development Opportunities Offered, as Indicated by Grant Managers (N = 40) 



What kinds of professional development opportunities have 
been offered or made possible by the program to date 
(check all that apply): 


Overall 
(A =40) 


CBTG 
(n = 25) 


Educate/ 
Innovate 
(« = H) 


eLearning 
Project 
(n = 1) 


LNV 
(» = 1) 


RWDC 
(« = 1) 


VTVLC 
in = 1) 


On-site session with a consultant (i.e., an outside expert who 
instructs and guides teachers in learning new processes and 
skills) 


70.0% 


60.0% 


81.8% 


X 


X 


X 


X 


In-school training (i.e., learning knowledge and skills from and 
with school faculty) 


70.0% 


60.0% 


81.8% 


X 


X 


X 


X 


Workshop (i.e., an interactive session or sessions facilitated by 
an expert and focused on practical knowledge and skills) 


62.5% 


56.0% 


63.6% 


X 


X 


X 


X 


One-on-one or group training with technology coordinators or 
aides 


42.5% 


32.0% 


45.5% 


X 


X 


X 


X 


On-demand, online, or Web-delivered professional 
development 


32.5% 


24.0% 


36.4% 




X 


X 


X 


Institute (i.e., a focused training or planning session spanning 
several days or longer) 


25.0% 


20.0% 


27.3% 


X 




X 




Off-site conference provided by professional associations or 
organizations 


22.5% 


16.0% 


36.4% 


X 








Network of teachers outside the school or school district to 
discuss implementation 


20.0% 


8.0% 


36.4% 


X 


X 




X 



Table 16. Professional Development Events Attended by Teachers (N = 165; select all that apply) 



In which of the following professional development events did you participate 
this school year as part of your school’s Ed-Tech grant program (check all that 
apply): 


Overall 
(A= 165) 


CBTG 
in = 73) 


Educate/ 
Innovate 
(n = 48) 


eLearning 
Project 
(n = 19) 


LNV 
in = 18) 


Single day, multihour sessions (e.g., in-service and professional development days) 


52.7%, 


56.2% 


56.3% 


42.1% 


50.0% 


Coaching from an external consultant who comes to the school 


31.5% 


28.8% 


25.0% 


78.9% 


11.1% 


Coaching from a school staff member who shares relevant professional expertise 
(e.g., project-based learning and technology) 


46.7% 


53.4% 


54.2% 


47.4% 


16.7% 


Multiday sessions (e.g., institutes) 


28.5% 


26.0% 


29.2% 


36.8% 


27.8% 


Other 


18.8% 


17.8% 


12.5% 


15.8% 


27.8% 



Note. The overall column includes three respondents from RWDC and three respondents whose programs were not identified. 
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Professional Development Focus. The overwhelming majority of grant managers (90 percent) 
indicated that using technology for instruction and learning was the major emphasis of the 
professional development through their grant programs to date (see Figure 1). Additionally, more 
than one half of the grant managers surveyed reported that 21st century skills were a major 
emphasis of the professional development in their grant programs. 



Figure 1. Emphasis on Professional Development as Indicated by Grant Managers 

21 % ■ 



Team teaching 
Student assessment 
Differentiated instruction 
Project-based learning 
Inquiry-based instruction 
Experiential learning 
21st century skills 
Uses of technology for instruction and learning 



29% 



I 



58 % 21 % 








43 % 29 % 


) 




1 






\ 4 % 33 % 










l Minor Emphasis Major Emphasis 
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Percent of Teachers 



80% 
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Interviews with staff at multiple sites per program provided further detail about the typical 
formats and foci of professional development: 

• CBTG Program. The most common type of professional development was one-on-one 
or small-group coaching, facilitated by district and school technology integration 
specialists. Other types of professional development included in-service training, self- 
study, workshops, classroom observations, and graduate courses. Professional 
development helped teachers learn how to use the acquired equipment and integrate it 
into their curricula and class instruction. Overall, the events varied in duration, ranging 
from workshops of a few hours to conferences lasting 2 to 3 days. Across all schools, 
staff typically spent a total of about three days participating in professional development. 

• Educate/Innovate Program. Interviewees from four sites indicated that professional 
development events varied in length and were facilitated by a variety of trainers, district- 
based staff, outside consultants, and content experts. Like the CBTG Program, these 
sessions focused on content-area topics, the use of technology and applications, and the 
management of technology in the classroom (including student and teacher responsibility 
and policies around taking devices home). Professional development for grantee teachers 
occurred regularly throughout the school year and ranged from 1.5 -hour biweekly 
sessions to 2.5-hour monthly sessions. Staff typically spent a total of 2 to 3 days 
participating in professional development. 
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• eLearning Project. Each eLearning Project team attended a 3 -day institute, where 
faculty from the University of Vermont-Montpelier demonstrated technology integration 
and participants had the opportunity to plan and share curriculum resources. Each site 
also worked with eLearning Project coaches on an ongoing basis. The schools reported 
that coaches visited their schools approximately 3 days per month and were also available 
for e-mail and telephone consultation. Interviewees from all schools also reported 
working with coaches in the classroom to initiate new teaching methods that further 
integrated technology into instruction. In addition to the eLearning Project institute and 
on-site coaching, the schools reported other professional development activities of 
varying lengths, ranging in duration from a few hours to semester-long courses. These 
opportunities included in-service days and short workshops after school hours with 
colleagues, self-study, and graduate level university courses. Finally, participants at three 
sites indicated that they observed technology integration initiatives in classrooms at other 
schools as part of the institute’s program. In total, participants in the eLearning Project 
received 6 to 10 days of professional development. 

• LNV. Between two and four teachers from each school received professional 
development in the form of 2- to 3-hour workshops held with their LNV equipment. The 
workshops were hosted by the program’s fonner director and provided information on 
how to use LNV equipment and the CILC database. These workshops enabled teachers to 
become technically proficient in using the videoconferencing equipment and the CILC 
website. Two teachers also indicated that the workshops provided an opportunity to 
refine their skills in searching the CILC database and better match programs with their 
curriculum needs. Additionally, three teachers received ongoing technical assistance from 
a technology specialist at one school. 

These findings from the interviews indicate that for three programs (the CBTG Program, the 
Educate/Innovate Program, and the eLearning Project), the focus of professional development 
was on the integration of technology with classroom instruction. The professional development 
in these programs appears to have been sustained — that is, occurring over several months. It also 
appears to be closely tied to the teaching of specific content standards. Whereas the CBTG 
Program and the Educate/Innovate Program involved approximately 3 days of professional 
development, the eLearning Project involved 6 to 10 days per participant. LNV differed from the 
others in regard to its focus, which was more on technical proficiency with the use of the 
videoconferencing equipment. 

2b. What Is the Scope of Participation in Professional Development? 

The scope of participation in professional development was considered in tenns of the number of 
teachers participating and the intensity of participation. Both grant managers and teachers 
responded to survey questions on the extent of teacher and staff participation in Ed-Tech grant 
program activities over the 2010-11 school year. Grant manager survey responses indicated that 
across all programs, as shown in Table 17, participation in professional development events was 
typically 10 or fewer participants. This is evidence of full participation for the eLearning Project 
and the Educate/Innovate Program, which typically included between 1 and 10 participants. 
About one half of the managers of the CBTG Program reported teacher participation in the range 
of 1 to 5. 
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Grant managers across the four grant programs involved in the interviews — the eLearning 
Project, the Educate/Innovate Program, the CBTG Program, and LNV — stated that the number 
of teachers participating in professional development, and their duration of participation, was 
consistent with program expectations. Because the grant manager responses in Table 17 reflect 
the grantwide level of participation, findings from the interviews with LNV and eLearning 
Project grant managers and teachers clarified the typical number of participants per site: 

• The LNV grant manager indicated that participating LNV schools typically had 2 to 4 
teachers participate in the professional development. 

• At each of the four eLearning Project sites where interviews were conducted, between 5 
and 7 teachers participated on the project team. At three of these schools, the entire 
teaching staff participated in on-site professional development offered by the external 
coach. 

The extent to which teachers and other program staff participated in professional development 
activities can be measured by the number of hours spent attending professional development 
events offered to date. According to both grant managers and teachers, the number of hours spent 
attending professional development events varied by grant program. As Table 18 shows, most of 
the grant managers for the CBTG Program (44 percent) reported that participants in their 
programs attended between 1 and 10 hours of professional development, while the majority of 
Educate/Innovate grant managers (55 percent) reported that participants spent between 1 1 and 
20 hours attending professional development. These findings are similar to those reported in 
2009-10, with the exception that no grant managers reported that teachers had not attended any 
professional development in 2009-10. This finding suggests that all the grant programs are 
moving forward with their implementation plans. 
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Table 17. Frequency of Total Number of Professional Development Participants to Date as Indicated by Grant Managers 



What is the total number of individuals who have 
participated in professional development activities to 
date? 


Overall 
(A = 34) 


CBTG 
G* = 21) 


Educate/ 
Innovate 
(« = 9) 


eLearning 
Project 
(n = 1) 


LNV 
in = 1) 


RWDC 
in = 1) 


VTVLC 
in = 1) 


1 to 5 


38 . 2 % 


52.5% 


33.3% 










6 to 10 


23 . 5 % 


23.9% 


11.1% 






X 




11 to 15 


8 . 8 % 


14.4% 


0.0% 










16 to 20 


5 . 9 % 


0.0% 


22.2% 










More than 20 


20 . 6 % 


9.6% 


22.2% 


X 


X 




X 


Unsure 


2 . 9 % 


0.0% 


11.1% 











Note. Not all columns total 100.0% due to rounding. 



Table 18. Frequency of Total Estimated Hours of Professional Development Attended 
Per Participant to Date as Indicated by Grant Managers 



What is the total estimated number of hours of 
professional development provided to date per 
participant? 


Overall 
(A = 40) 


CBTG 
in = 25) 


Educate/ 
Innovate 
in = 11) 


eLearning 
Project 
in = 1) 


LNV 
in = 1) 


RWDC 
in = 1) 


VTVLC 
in = 1) 


1-10 hours 


67.7% 


44.0% 


27.3% 




X 


X 




1 1-20 hours 


14 . 7 %, 


20.0% 


54.5% 










21-30 hours 


11 . 8 % 


20.0% 


0.0% 










3 1 hours or more 


11 . 8 % 


16.0% 


18.2% 


X 






X 



Note. The overall column includes three respondents from RWDC and three respondents whose programs were not identified. 
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CBTG, Education/Innovate, and LNV teachers responded similarly to the survey question 
regarding the number of hours spent attending professional development events, with the vast 
majority of the respondents indicating that they had received 20 hours or less of professional 
development to date (Table 19). Respondents for the eLearning Project reported the most hours 
spent attending professional development, with nearly one third of the respondents indicating 
that they spent 41 hours or more. 



Table 19. Frequency of Total Estimated Hours of Professional Development 
Attended to Date as Indicated by Teachers 



Approximately how many total hours did 
you participate in the events attended 
during the 2010-11 school year that you 
indicated above? 


Overall 
(A =159) 


CBTG 
(« = 72) 


Educate/ 
Innovate 
(n = 46) 


eLearning 

Project 

(» = 17) 


LNV 
(» = 17) 


1-10 hours 


27 . 0 % 


30.6% 


17.4% 


5.9% 


58.8% 


1 1-20 hours 


32 . 7 % 


27.8% 


50.0% 


23.5% 


17.6% 


21-30 hours 


16 . 4 % 


13.9% 


21.7% 


23.5% 


5.9% 


31-40 hours 


10 . 1 % 


9.7% 


4.3% 


17.6% 


11.8% 


4 1 hours or more 


13 . 8 % 


18.1% 


6.5% 


29.4% 


5.9% 



Note. The overall column includes three respondents from RWDC and three respondents whose programs were not identified. 



In summary, grant managers indicated that the level of participation in professional development 
met program expectations. As indicated in the survey findings, the number and duration of 
participation varied across programs, most notably with the greater duration of professional 
development for eLearning Project teachers. This reflects the different structure of this program. 
The extent to which participants in grant programs that began during the 2009-10 school year 
attended professional development events during the 2010-11 school year, as compared to the 
amount of professional development offered during the 2009-10 school year provides evidence 
that these grant programs have moved into the later stages of implementation. 

2c. What Are Participants’ Opinions of the Quality and the Effectiveness of Professional 
Development? 

Through the teacher survey, respondents were asked to rate their agreement on statements related 
to the usefulness and the effectiveness of the professional development they attended and 
ultimately its impact on their teaching practices. 

Overall, as shown in Table 20, the attendees of professional development events indicated that 
the quality of the event was either excellent or good across categories. Additionally, nearly one 
half of the teachers reported that the professional development events they attended were 
excellent when it came to providing resources and tools that could be applied in the classroom. 
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Table 20. Quality of Professional Development as Indicated by Teachers 



How would you rate the professional 
development in terms of the following? 


N 


Poor or 
Inadequate 


Barely 

Adequate 


Good 


Excellent 


Provided resources and tools 1 can use in the 
classroom 


158 


1.9% 


3.8% 


44.9% 


49.4% 


Connected to the goals of your Ed-Tech grant 


157 


2.5% 


5.1% 


49.0% 


43.3% 


Clarity of content presented 


158 


1.3% 


1.9% 


60.8% 


36.1% 


Helped me understand my role and 
responsibilities in implementing this program at 
my school 


158 


3.8% 


10.1% 


55.1% 


31.0% 


Organization 


158 


1.3% 


3.8% 


64.6% 


30.4% 


Addressed the needs of the students in my 
classroom 


158 


1.9% 


7.6% 


63.3% 


27.2% 



Note. Not all rows total 100.0% due to rounding. 



There were some differences in the ratings of professional development quality by program. 
Because these differences persisted across the six aspects of quality, they are summarized in 
Table 21 by taking the average percentage of teachers selecting each rating for each program. On 
average (across all six items and all programs), 56 percent of the teachers rated professional 
development as good, and 36 percent of the teachers rated it excellent. Teachers from the 
eLearning Project were more likely to rate the quality as good (77 percent) than excellent 
(18 percent). See Appendix B, Table 1 for a full breakout of responses by program. 



Table 21. Average Frequency of Teacher Rating of Professional Development 
Across Five Items by Grant Program 





Overall 
(A =158) 


CBTG 
(« = 72) 


Educate/Innovate 
{n = 46) 


eLearning Project 
in = 17) 


LNV 
in = 17) 


Poor or inadequate 


2.1% 


0.9% 


0.7% 


0.0% 


2.0% 


Barely adequate 


5.4% 


5.4% 


5.8% 


4.9% 


6.9% 


Good 


56.3% 


55.0% 


51.5% 


76.5% 


58.8% 


Excellent 


36.2% 


38.8% 


42.1% 


18.6% 


32.4% 



Note. The overall column includes three respondents from RWDC and three respondents whose programs were not identified. 



Emphasis and Impact of Professional Development on Teacher Practice. Teachers were 
asked to indicate the level of impact that their professional development had on their teaching 
practice. As shown in Table 22, nearly three fourths of the respondents (72 percent) across grant 
programs reported that their participation had a noticeable to very strong impact on their teaching 
practice. When the results of the survey are disaggregated by grant program, participants from 
the eLearning Project reported the most impact on their teaching practices (94 percent indicated a 
noticeable to very strong impact). 
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Table 22. Teacher Ratings of Impact of Professional Development 
on Instructional Practice (N = 159) 



To what extent did attendance at the 
professional development events indicated 
above make an impact on your teaching 
practice? 


Overall 
(N = 159) 


CBTG 
(» = 72) 


Educate/ 
Innovate 
C n = 46) 


eLearning 

Project 

(» = 17) 


LNV 
(n = 17) 


No impact 


4.4% 


2.8% 


2.2% 


0.0% 


11.8% 


Slight impact to date 


21.9% 


25.0% 


26.1% 


5.9% 


35.3% 


Noticeable impact 


50.9% 


50.0% 


56.5% 


70.6% 


23.5% 


Very strong impact 


20.8% 


22.2% 


15.2% 


23.5% 


29.4% 



Note. The overall column includes three respondents from RWDC and three respondents whose programs were not identified. 



Overall, professional development was perceived as effective by interviewees from the four 
grant programs (the eLearning Project, the Educate/Innovate Program, the CBTG Program, and 
LNV) in terms of helping them carry out their grant goals. Across programs, teachers stated that 
their hands-on training helped them learn how to integrate technology into their curricula. For 
example, one CBTG teacher noted, “The grant paid for a three-credit course on using the smart 
board, and that was really important to the implementation. I couldn’t be implementing it without 
that course.” Specific findings for each program included the following: 

• CBTG Program. At three of the schools, a district technology integration specialist 
worked directly with teachers on how to incorporate technology into their specific 
curricula. In contrast, teachers at one school noted that the professional development was 
not closely aligned to the targeted curriculum. 

• Educate/Innovate Program. All four Educate/Innovate teachers interviewed indicated 
that professional development sessions for their grant programs were helpful. In 
particular, these sessions encouraged teachers across schools to collaborate and share 
their experiences. Teachers also reported learning protocols for managing technology in 
the classroom and at home; gaining skill with the use of devices; and increasing their 
understanding of technology integration practices in general and their content area in 
particular. 

• eLearning Project. Respondents from all schools considered the various professional 
development opportunities to be valuable for the success of their respective projects. 

Most of these comments focused on the eLearning Project coach, who helped teachers set 
up and use hardware and software. The coaches also addressed teachers’ specific needs 
for technology integration, such as augmenting existing curricula, aligning curricula to 
technology standards, and troubleshooting equipment or other issues. Two coaches built 
Google sites as a forum for team blogs and resources, and one coach helped teachers 
better use technology staff resources at their schools. All teams described their coaches as 
being a sounding board for questions and responsive to their needs. Team members in 
three schools stated that their coaches were an effective resource. In one school, however, 
team members felt that the coach’s role was less critical because of the principal’s strong 
leadership for the program. 

• LNV. Most of the teachers indicated that the workshops enabled them to become 
technically proficient in using LNV equipment and the CILC website. Two teachers also 
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indicated that the workshops provided an opportunity to refine their skills in searching 
the CILC database and better match programs with their curricular needs. 

In summary, the participants expressed overall satisfaction with the quality of professional 
development provided and indicated that their attendance at professional development events had 
a noticeable to very strong impact on their teaching practices. Teachers from the eLearning 
Project reported a similar quality of professional development as teachers from the other grant 
programs but greater impact overall. 

2d. To What Extent Are Teachers Provided Opportunities to Collaborate on Implementing 
Program Objectives for Technology Integration? 

The creation of professional learning networks is one of Vermont’s statewide Ed-Tech strategies. 
Teacher collaboration is a critical element in supporting the work of teachers. As reported earlier 
(see Table 9), nearly two thirds of the grant managers indicated that teacher collaboration was a 
major focus of their grant programs during the 2010-11 school year. On surveys and in 
interviews, teachers were asked to report the frequency with which they had opportunities to 
work with and learn from other teachers this year as part of their Ed-Tech grant programs. 
Teachers were also asked to indicate the importance of collaboration for achieving the goals of 
their grant programs, as well as the frequency of their collaboration in comparison to the 
previous school year. 

Overall, teachers frequently collaborated with their colleagues. Major survey findings across 
programs, as summarized in Table 23, are as follows: 

• The majority of the teachers (69 percent) indicated that they collaborate within their 
grade or team level either daily/almost daily (38 percent) or weekly (3 1 percent). 

• More than one third of the teachers (37 percent) collaborated with colleagues outside 
their grade level or subject area either daily/almost daily (10 percent) or weekly 

(28 percent). 

• A minority of the teachers (10 percent) collaborated with colleagues outside their school 
daily/almost daily (1 percent) or weekly (12 percent). 



Table 23. Frequency of Collaboration as Reported by Teachers 



Over the past school year, how 
often have you collaborated with 
other teachers: 


N 


Not at 

All 


2-3 Times per 
Semester 


Monthly 


Weekly 


Daily/ 

Almost 

Daily 


Within your grade-level team or 
subject area? 


149 


4.0% 


10.7% 


16.1% 


30.9% 


38.3% 


Outside your grade-level team or 
subject area, but within your school? 


148 


10.1% 


26.4% 


25.7% 


27.7% 


10.1% 


Outside your school (e.g., through an 
online network)? 


146 


32.9% 


34.9% 


19.2% 


11.6% 


1.4% 



These findings varied somewhat by program (see Appendix B, Table 2). 
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• Teachers from the eLearning Project reported the highest levels of grade- or team-level 
collaboration, with 88 percent reporting that they collaborate within their grade or team 
level either daily/almost daily or weekly (44 percent for both). 

• LNV respondents more frequently collaborated with teachers outside their schools — 

43 percent indicated that they do so either daily/almost daily (7 percent) or weekly 
(36 percent). 

These findings are consistent with the structures and the purpose of each program, with the 
eLearning Project emphasizing teamwork, and LNV emphasizing connections to outside 
resources. 

Interviews with teachers from four of the grant programs provided further detail about their 
opportunities to collaborate. Across all programs and sites, teachers reported opportunities to 
collaborate during common planning periods and grade-level meetings. Across programs, 
teachers commonly reported informal opportunities to collaborate, such as when teachers 
exchange ideas throughout their day (such as during lunch breaks). Respondents from all the 
eLearning Project sites reported additional opportunities to collaborate, such as during scheduled 
team meetings with the eLearning Project coach. Several Educate/Innovate Program sites held 
monthly or weekly training sessions that provided teachers from several schools with a forum for 
sharing and troubleshooting. As one teacher noted, 

[During our professional development sessions, we collaborate on] what [we] are doing 
[with students in the classroom and] what resources [we] have used, because two of the 
other schools [involved in our grant] are using iPads as well. [We also talk about] 
troubleshooting [these devices]. 

The effectiveness of these opportunities varied across sites within programs. At two of the four 
CBTG and Educate/Innovate grant sites, teachers reported that opportunities to collaborate are 
sufficient to support the goal of technology integration. However, at the other two sites from 
each of these programs, some teachers stated that these opportunities are not sufficient, and 
additional collaboration time dedicated to technology integration is needed. All four eLearning 
Project sites visited in spring 2011 reported that opportunities to collaborate were sufficient. 

Impact of Program on Collaboration. The survey asked teachers about the impact their grant 
programs had on collaboration. The majority (68 percent) reported that their grant programs have 
had a noticeable (46 percent) to very strong (22 percent) impact on their level of collaboration 
with colleagues, as shown in Table 24. 
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Table 24. Teacher Rating of Impact of Grant Program on Level of Collaboration 



Overall, how much of an impact has your 
Ed-Tech grant program had on your level 
of collaboration with colleagues? 


Overall 
(N = 148) 


CBTG 
(n = 
67) 


Educate/ 
Innovate 
(n = 45) 


eLearning 

Project 

(» = 17) 


LNV 
0 n = 13) 


No impact/negative impact 


7.4% 


4.5% 


6.7% 


5.9% 


7.7% 


Slight impact to date 


25.0% 


23.9% 


24.4% 


23.5% 


38.5% 


Noticeable impact 


45.9% 


47.8% 


42.2% 


58.8% 


38.5% 


Very strong impact 


21.6% 


23.9% 


26.7% 


11.8% 


15.4% 



Note. The overall column includes three respondents from RWDC and three respondents whose programs were not identified. 



Teachers were asked to compare the frequency of their collaboration during the current school 
year to that of the 2009-10 school year. As summarized in Table 25, 61 percent of the teachers 
reported that compared to last year, they collaborate with other teachers more (5 1 percent) or 
much more frequently (10 percent). eLearning Project respondents reported the greatest impact 
on the frequency of collaboration, with 77 percent of the respondents reporting that they 
collaborated more (7 1 percent) or much more frequently (6 percent) during the current year. 



Table 25. Frequency of Collaboration by Grant Program as Indicated by Teachers 



Compared to last year, how frequently do 
you collaborate with other teachers? 


Overall 
(N= 155) 


CBTG 
(» = 71) 


Educate/ 
Innovate 
(n = 46) 


eLearning 

Project 

(» = 17) 


LNV 
(n = 15) 


Less frequently 


5.2% 


4.2% 


0.0% 


5.9% 


0.0% 


About the same 


36.1% 


39.4% 


32.6% 


17.6% 


40.0% 


More frequently 


51.0% 


45.1% 


58.7% 


70.6% 


46.7% 


Much more frequently 


9.7% 


11.3% 


8.7% 


5.9% 


13.3% 



Note. The overall column includes three respondents from RWDC and three respondents whose programs were not identified. 



Impact of Collaboration on Grant Goals. Teachers were also asked to indicate the importance 
of collaboration for achieving the goals of their grants. Overall, the respondents reported that 
collaboration with other teachers and school staff has been either very important (49 percent) or 
moderately important (36 percent) for accomplishing the goals of their grants (Table 26). While 
respondents across programs were generally positive about the importance of collaboration, LNV 
respondents overwhelmingly reported that collaboration was very important for achieving the 
goals of their grant (73 percent). 



Table 26. Teacher Rating of Importance of Collaboration by Grant Program (N = 155) 



How important has collaboration with 
other teachers and school staff been for 
accomplishing the goals of your grant 
program? 


Overall 
(N= 155) 


CBTG 
(n = 71) 


Educate/ 
Innovate 
(n = 46) 


eLearning 

Project 

(» = 17) 


LNV 
o n = 15) 


Not at all important 


2.6%> 


4.2% 


0.0% 


0.0% 


0.0% 


Minimally important 


12.3%o 


16.9% 


13.0% 


0.0% 


0.0% 


Moderately important 


36.1% 


33.8% 


39.1% 


41.2% 


26.7% 


Very important 


49.0% 


45.1% 


47.8% 


58.8% 


73.3% 



Note. The overall column includes three respondents from RWDC and three respondents whose programs were not identified. 
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Interviews with teachers confirmed that collaboration was important for accomplishing the grant 
goals. Across all programs, teachers consistently noted that their peer support system was 
essential for sharing ideas, resources, and strategies on technology integration and developing 
lesson plans and student projects. Teachers also helped each other leam how to use equipment, 
how to best use devices in the classroom with students, and how to troubleshoot issues that came 
up with technology. 

In the interviews, teachers also revealed that a key barrier faced by participants from all grant 
programs was a lack of time for teachers to collaborate, leam, and develop technology-rich 
lesson plans. As one teacher explained, 

Time [is] always [a] huge [barrier] for everyone. There are always millions of priorities, 
and [although] there’s no lack of motivation, [there’s a lack of] time to reach out to the 
other colleagues to share what they’re working on. 

In summary, grant managers and teachers indicated that teacher collaboration is a considerable 
focus of the Ed-Tech grant programs. Teachers are frequently collaborating on technology within 
grade-level teams or subject areas and to a lesser extent outside their grade level, subject, and 
school. Teachers report that the grant programs have made a noticeable impact on opportunities 
for collaboration, although some say they could use more. Teachers also reported that 
collaboration has been important for accomplishing the overall goals of the individual grants. 

2e. To What Extent Do Administrators Support, Advocate, 
and Encourage Technology Integration? 

On the survey, teachers indicated which statements about administrative support were true for 
their programs. The response frequencies to these items are summarized in Figure 2. Across all 
grant programs, over 70 percent of the respondents reported that their principals understood the 
instructional goals of their grant programs and actively encouraged the adoption of instructional 
goals; slightly more than one half reported that, in addition, their principals hold them 
accountable for adopting the instructional goals of the grant programs. Few participants 
(13 percent) indicated that their principals are not aware or do not understand the program. 
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Figure 2. Percent of Teachers Agreeing with Statements 
About Administrative Direction and Support 




My principal 
understands the 
instructional goals of 
my grant program. 
(N=143) 



My principal actively 
encourages me to 
adopt the instructional 
goals of my grant 
program. (N=142) 



My principal holds me 
accountable for 
adopting the 
instructional goals of 
my grant 
program. (N=142) 



My principal does not 
understand or is not 
aware of the 
instructional goals of 
my grant program. 
(N=143) 



Interview data across the four grant programs (the eLearning Project, the Educate/Innovate 
Program, the CBTG Program, and LNV) show an overall high level of administrative 
involvement and support for grant implementation. There are two broad categories of support. 
First, school leaders provide resources, such as funds for additional equipment or professional 
development, scheduled planning time, and substitute teachers to cover teacher participation in 
grant activities. Second, school leaders provide leadership for technology integration. Principals 
express this leadership in several ways, including supporting and encouraging teachers, setting 
expectations related to technology integration, recognizing teacher and student accomplishments 
related to technology, encouraging teachers to share their progress at staff meetings, and 
modeling the use of technology in their own work as principals. As one eLearning Project 
teacher noted, “[Our principal] has expectations that all classrooms will integrate technology. 
[It’s] a requirement this year that teachers create a technology goal that is measurable, and he’s 
keeping documentation of our ability to meet the goals that we created.” 

The interviews indicated that the level of administrative support and guidance varied within each 
grant program: 

■ CBTG Program. The four schools visited received varying levels of administrative 
support and guidance. Administrative support at all four schools included scheduled 
planning time and general encouragement from the principal to try new practices with 
technology. At one site, the teachers reported high principal involvement, in the fonn of 
specific teaching goals and expectations for technology integration. In the remaining 
schools, principals provided minimal or unclear guidance or communication about 
expectations for changes in practice. 
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■ eLearning Project. In the four sites visited, staff indicated that principals provide strong 
leadership for fostering a school culture that institutionalizes technology integration into 
everyday practice. All grant programs noted that their principals were supportive of this 
grant program and provided teachers with needed resources. Moreover, in all sites, the 
principal was a member of the eLearning Project team and participated in its regular 
meetings. There were some variations among sites in tenns of principal leadership. At 
two sites, principals set expectations for technology integration and followed up with 
teachers on these expectations through classroom observations. At the other two sites, the 
setting of expectations and follow-up was minimal. 

■ Educate/Innovate Program. The interviewees from this program reported that 
principals typically have limited direct involvement in grant implementation, although 
they provide teachers with time and resources to carry out the grant work. This includes 
finding additional funds for teachers to attend statewide conferences and pay for 
substitute teachers. Interviewees reported that technology directors have demonstrated 
strong leadership in coordinating and facilitating the grant programs at schools. One 
teacher described the support provided by the technology coordinator: 

Our district [technology director] has been incredible. [S/he’s] always available [for 
teachers and] s/he [facilitates our grant] meetings. [S/he’s] also noticed how well [grant 
activities have flourished] here with the use of technology and [S/he has] encouraged 
other schools to follow what we've done or [has set up for schools to] come and visit. 

■ LNV. All teachers indicated that school administrators are aware of the LNV equipment 
and credits to access the CILC content, and four of six teachers specifically felt 
encouraged by administrators to try new practices with the LNV program. Two of six 
teachers noted that their principals allow for time to share and plan with other teachers 
about using the LNV program. The respondents did not indicate that principals had 
specific expectations regarding the use of LNV resources. One teacher explained that the 
LNV program is teacher driven. 

2f. To What Extent and From What Sources Do Teachers Receive Technology Support? 

Through the survey, teachers were asked to indicate the various barriers and challenges faced in 
implementing their grant programs. Of the challenges that the teachers indicated, as shown in 
Figure 3, the majority reported that limited support for technology was either a minimal 
challenge (34 percent) or did not pose as a challenge (30 percent). Although most teachers 
reported that a lack of technology support did not pose as a barrier to the implementation of their 
Ed-Tech grant, slightly more than one third of the respondents indicated that limited technology 
support was a moderate challenge (20 percent) or a minimal challenge (16 percent). 
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Figure 3. Teacher Ratings of the Extent to Which Limited Support for Technology 
Posed a Barrier to Grant Implementation (N = 144) 




Interviews with teachers clarified the sources and types of technical support available. Teachers 
from across the four grant programs (the eLearning Project, the Educate/Innovate Program, the 
CBTG Program, and LNV) seem to have adequate access to district or school technology 
specialists to carry out this grant work. As one teacher from a CBTG Program explained, “Our 
technology integrationist has been side by side with us through and through. We have our tech 
people, our technology director and his assistants, [and] our librarian [that support our grant 
work].” The grant teams involved with the CBTG Program, the Educate/Innovate Program, and 
the eLearning Project usually include a technology support specialist. For example, an 
Educate/Innovate site paired a specialty teacher with a technology integration specialist to carry 
out their grant work, and all eLearning Project teams include their coach and technology staff. 
Experienced and skilled teachers also provide technology support to their colleagues. As a 
principal explained, “One of our teachers at the middle [school] is incredibly innovative in 
technology [and] is spending half of his time supporting teachers [in technology integration].” 

Summary 

Professional development was offered in several different formats. The most commonly offered 
professional development events were single-day, multihour trainings, usually in the form of an 
on-site session with a consultant. Participants described the level and the quality of support for 
technology integration related to their grants. In regard to professional development, most of the 
grants involved fewer than 10 participants and 20 or fewer hours of professional development. 
However, there was substantial variation across programs, most notably with the greater duration 
of professional development for eLearning Project teachers. 
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Participants were satisfied with the quality of professional development and indicated that their 
attendance at professional development events had a noticeable to very strong impact on their 
teaching practice. Teachers from the eLearning Project reported somewhat greater impact of 
their professional development. 

Grant managers and teachers indicated that teacher collaboration is a considerable focus of the 
Ed-Tech grant programs. Teachers frequently collaborate on technology within grade-level 
teams or subject areas and, to a lesser extent, outside their grade level, subject, and school. 
Teachers reported that the grant programs have made a noticeable impact on opportunities for 
collaboration, and that collaboration has been very important for grant implementation. 

Teachers reported high levels of principal awareness and encouragement of technology 
integration related to the grant. Moreover, a majority of teachers indicated their principals held 
them accountable for the instructional goals of the grant. The interviews largely reflected this 
finding and illustrated the various ways that principals provide both resources and leadership for 
technology integration. The extent of principal involvement varied somewhat depending on the 
type of program (with the eLearning Project having the highest involvement). In some instances, 
the district technology coordinator provided guidance rather than the principal. Teachers 
typically received sufficient technical support to accomplish the goals of their grant programs. 

Question 3: Do the Ed-Tech Grant Programs Promote Technology Integration 
in Support of Student-Centered Learning? 

This evaluation question addresses the impact of the program on teachers’ instructional and 
assessment practices and, in turn, on the learning experiences that are available to students. The 
impact on instructional practice is discussed in terms of student-centered instructional strategies 
(e.g., project-based learning and collaborative work), technology integration, and assessment. 
This subsection concludes with teacher ratings of the program’s overall impact on teaching 
effectiveness. 

Impact on Instructional Practices 

Teachers rated the extent of the impact of the program on a variety of different instructional 
practices related to technology integration. As summarized in Table 27, about 66 percent of the 
teachers typically reported that their Ed-Tech grant programs had a noticeable (46 percent) or 
very strong (19 percent) impact on their instructional practices. Respondents from the CBTG 
Program, the Educate/Innovate Program, and the eLearning Project reported similar levels of 
impact — 7 1 percent, 7 1 percent, and 74 percent, respectively, reported a noticeable or very 
strong impact on their instructional practices. In contrast, nearly three fourths of LNV 
respondents indicated that their professional development has had no impact (6 percent), a slight 
impact (44 percent), or that the question was not applicable (22 percent). 
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Table 27. Teacher Ratings of Extent to Which Participation in Ed-Tech Program 
Has Made an Impact on Instructional Practices 





Overall 
(N= 146) 


CBTG 
(n = 73) 


Educate/ 
Innovate 
(if = 48) 


eLearning 
Project 
(if = 19) 


LNV 

(if = 18) 


No impact 


3.6% 


4.1% 


2.1% 


0.0% 


5.6% 


Slight impact to date 


22.4% 


17.8% 


22.9% 


15.8% 


44.4% 


Noticeable impact 


46.1% 


47.9% 


52.1% 


47.4% 


27.8% 


Very strong impact 


19.4% 


23.3% 


18.8% 


26.3% 


0.0% 


Not applicable 


8.5% 


6.8% 


4.2% 


10.5% 


22.2% 



Note. The overall column includes three respondents from RWDC and three respondents whose programs were not 
identified. 



Project-Based Learning. One of the practices for which teachers typically reported a noticeable 
or very strong impact was providing opportunities for students to work on extended projects. 
Teachers were also asked to indicate the number of project-based assignments (spanning more 
than a week in duration) that they assigned during the 2010-11 school year, as well as the 
number they recall having assigned during the previous school year. As shown in Table 28, 
across all programs, the respondents reported assigning an average of 5.8 projects in 2010-1 1 
and 4.2 projects during the previous year. Teachers in the CBTG Program reported the highest 
increase; they assigned an average of 5.6 projects in 2010-1 1 compared to 3.7 for the previous 
year. 



Table 28. Average Percent Change in Frequency of Project Based Learning (N = 126) 



Grant Program 


N 


Average Number of Projects 
Assigned During the 2009-10 
School Year 


Average Number of Projects 
Assigned During the 2010-11 
School Year 


CBTG 


55 


3.7 


5.6 


Educate/Innovate 


40 


4.9 


6.2 


eLearning Project 


16 


4.3 


5.8 


LNV 


10 


4.5 


5.4 


Overall 


121 


4.2 


5.8 



Interviews with teachers across the four programs (the eLearning Project, the Educate/Innovate 
Program, the CBTG Program, and LNV) provide further detail about the ways in which the 
program promoted a shift in teaching pedagogies and practices. As an overall finding, teachers 
report that their technology programs have increased their offering of learning opportunities that 
are personalized, collaborative, and differentiated. 

Personalized Learning. Interviews from the eLearning Project and Educate/Innovate grants 
suggest that with technology integration, teachers have increased their use of projects that are 
relevant to students’ personal lives, interests, families, and communities. For example, students 
from other countries who are English language learners (ELLs) piloted the use of flip and digital 
cameras and voice-recording devices with the ELL teacher at their schools to produce narrated 
video clips about their families and cultures. These ELL students also created safety-oriented 
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public service announcement videos for their peers and the larger community on topics relevant 
to students’ interests, such as bicycle safety. 

Student Collaboration. Students from the eLearning Project, the Educate/Innovate Program, 
and the CBTG Program often collaborate through tools such as e-mail, blogs, live chat forums, 
and Google Docs. This technology allows students to collaborate more efficiently without having 
to be in the same place. Because students have become experts in using their technological 
devices and software programs, they help teach each other and oftentimes help teachers 
troubleshoot technological issues. As one teacher explained, “[Aspects] of this [project] have 
been very successful. I’m pleased with student collaboration [and that] some students became 
experts in certain aspects or were able to help each other. It wasn’t all up to me.” Another teacher 
commented that because technology-rich projects engage students in kinesthetic and creative 
learning, students of all achievement levels have been able to demonstrate their skills and 
contribute to collaborative tasks. As a final example, the LNV equipment has increased 
opportunities for student collaboration by providing a forum for engaging in two-way, interactive 
learning and dialogue with content experts and their peers from other communities. 

Differentiated Instruction. Teachers interviewed from the CBTG Program, the 
Educate/Innovate Program, and the eLearning Project commented how technology integration 
supports differentiated and individualized learning. They noted that students can work at their 
own pace and ability level more easily when working with technology. For example, students at 
a CBTG school used Google Maps, voice thread, and iMovie to develop a narrated video clip of 
a world explorer’s journey. As one teacher noted, students who finished parts of the project 
ahead of other students could add on to their project by working with more advanced technology 
rather than filling their time with “busy work.” A teacher from the Educate/Innovate program 
explained, “[Through technology] everybody can contribute. [Technology] helps [our students] 
be engaged in different ways, depending on what works for them.” Echoing this sentiment, 
another teacher commented, “Different students have different learning styles. [I feel] 
technology addresses all of those learning styles. There’s something for everybody.” 

In summary, teachers across programs, with the exception of teachers participating in LNV, 
indicated that their grant programs have had a noticeable to strong impact on teaching practices 
related to technology integration. In particular, teachers reported an increase in the number of 
student projects they assigned this year compared to last. The programs promoted student- 
centered instruction in a variety of ways, including more personalized learning, student 
collaboration, and differentiation of instruction. 

Student Use of Technology 

The teacher survey asked teachers to report how frequently students used technology for 
different instructional purposes. Table 29 summarizes teacher responses across all the grant 
programs. For a full breakout of teacher responses by Ed-Tech grant program, see Appendix B, 
Table 3. As shown in Table 29, the most frequent purpose for student technology use in the 
classroom is additional practice or skill reinforcement, with 60 percent of the teachers stating 
that they employ it monthly (18 percent), weekly (34 percent), or daily/almost daily (26 percent). 
The next most frequent uses were online research, with 54 percent of the teachers assigning it 
monthly (22 percent), weekly (32 percent), or daily/almost daily (17 percent); and writing a 
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paper, with 48 percent indicating that they employ it monthly (23 percent), weekly (25 percent), 
or daily/almost daily (14 percent). 



Table 29. Purpose and Frequency of Student Use of Technology 



Over the past semester, how often did 
students in your classroom use 
technology for the following purposes 
during class time? 


N 


Not at 

All 


2-3 Times 
per 

Semester 


Monthly 


Weekly 


Daily/ 

Almost 

Daily 


Additional practice or skill reinforcement 


147 


6.8% 


15.0% 


18.4% 


34.0% 


25.9% 


Online research 


149 


14.1% 


15.4% 


21.5% 


32.2% 


16.8% 


Writing a paper 


149 


22.1% 


16.1% 


22.8% 


24.8% 


14.1% 


Online communication 


149 


28.2% 


14.1% 


20.8% 


24.2% 


12.8% 


Sorting/ categorizing information 


149 


28.2% 


24.8% 


28.9% 


12.8% 


5.4% 


Making a presentation 


147 


22.4% 


22.4% 


35.4% 


14.3% 


5.4% 


Drawing or creating visual art 


149 


37.6% 


24.2% 


22.8% 


10.7% 


4.7% 


Analyzing data 


149 


35.6% 


20.1% 


27.5% 


12.8% 


4.0% 


Composing or editing music 


149 


74.5% 


12.8% 


6.0% 


4.0% 


2.7% 


Creating or editing movies 


149 


51.7% 


29.5% 


12.8% 


5.4% 


0.7% 



Note. Not all rows total 100.0% due to rounding. 



The survey findings related to student purpose in using technology indicate that technology use 
by students is more frequent this year than the previous year, as summarized in Figure 4. The 
frequencies in the figure are a combination of the proportions of the teachers who selected daily, 
weekly, or monthly to describe the frequency of their use of technology for different purposes. 
For every purpose, there was an increase across years. 
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Percent of Teachers 



Figure 4. Percentage of Teachers Indicating Frequent Student Use of Technology 

for Different Purposes, by School Year 



100% 




2010 (N = 95) 

2011 (N = 149) 



Note: Total % indicating Daily, Weekly, or Monthly technology use for each purpose. 



The frequency of student use of technology varied by program. A full breakout of responses by 
program is provided in Appendix B, Table 3. 

• CBTG Program. Respondents from the CBTG Program reported less frequent student 
use of technology for online research than other programs, with 61 percent reporting 
monthly (25 percent), weekly (32 percent), or daily/almost daily (4 percent). This is less 
than the overall proportion of 7 1 percent. 

• Educate/Innovate Program. Respondents from the Educate/Innovate Program reported 
more frequent student use of technology for writing a paper, with 73 percent of the 
teachers selecting monthly (27 percent), weekly (27 percent), or daily/almost daily 

(20 percent). This is more than overall proportion of 61 percent. 

• eLearning Project. Respondents from the eLearning Project reported more frequent 
student use of technology for analyzing data and making a presentation. Fifty-nine 
percent of the eLearning Project teachers reported that their students used technology to 
analyze data monthly (29 percent) or weekly (29 percent), with none selecting 
daily/almost daily. This is greater than the overall proportion of 44 percent. Seventy-five 
percent of the eLearning Project teachers reported that their students use technology to 
make a presentation monthly (56 percent) or weekly (19 percent), with none selecting 
daily/almost daily. This is greater than the overall proportion of 55 percent. 

• LNY. When compared to the other grant programs overall, LNV teachers reported more 
frequent use of technology for sorting and categorizing infonnation, with 62 percent 
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reporting that they used technology for this purpose either monthly (23 percent), weekly 
(15 percent), or daily (23 percent), in comparison to 47 percent of teachers overall. 

Interviews with teachers provided further detail about the impact of the program on technology 
integration. Overall, the respondents indicate that teachers are integrating technology into the 
classroom on a more frequent and consistent basis. Teachers reported using more Web-based 
tools, such as various Google Apps, e-mail, and child-friendly search engines and resources. 
Students regularly use technological devices — such as mini laptops, netbooks, iPads, iPod Touch 
units, and flip and digital cameras — at a 2:1 or 1:1 device-to-student ratio. From the eLearning 
Project, the Educate/Innovate Program, and the CBTG Program, teachers reported giving 
students privileges to take devices out of the classroom to complete projects or homework with 
technology on their own time. These new or enhanced learning opportunities through technology 
integration were seldom or not available for students prior to grant funding. One principal from a 
CBTG school explained, 

[Our students feel] really fortunate and excited to be able to use [this new equipment], 
[which is] a huge reason why I wrote the grant. We live in a very rural part of the state. 
Most of our families live in high poverty areas [and] have no Internet connection. It’s 
imperative to me that [our students] have some exposure, experience, and time to develop 
[technological] skills so that by the time they get to middle and high school or beyond 
they’re not already years behind every[one] else. 

Interviewees from the LNV program described using the LNV technology infrequently, mainly 
because of the time needed to research appropriate programming and the limited amount of 
CILC credits available. However, when teachers used this equipment, they felt it enhanced the 
content of their existing curricula by providing supplemental or additional resources for students. 
In general, though, this grant- funded project does not seem to influence other types of 
technology integration that occur in these classrooms. 

In summary, teachers reported more frequent student use of technology during the current year 
than they had reported on a survey administered the previous year. Teachers most frequently 
used educational technology for additional practice or skill reinforcement; other common 
purposes for technology use were online research and paper writing. 

Impact on Assessment 

On the survey, teachers rated the extent of the impact of the program on four different 
assessment practices related to technology integration, and each teacher was categorized 
according to his or her typical response to these four items (see the Evaluation Methods for a list 
of the specific items). As summarized in Table 30, of the majority of teachers (52 percent) most 
typically reported that their Ed-Tech grant programs had a slight impact to date (43 percent) or 
no impact (9 percent) on their assessment practices. Teachers’ perception of their grant 
programs’ impacts on their assessment practices varied by program. Teachers from the CBTG 
Program reported the greatest impacts, with 60 percent indicating a noticeable (37 percent) to 
very strong (22 percent) impact. In contrast, 75 percent of the LNV respondents reported that 
their Ed-Tech grant programs have had no impact (42 percent) or a slight impact to date 
(33 percent). A full summary of responses to these items can be found in Appendix B, Table 4. 
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Table 30. Teacher Ratings of Extent to Which Participation in Ed-Tech Program Has 

Made an Impact on Assessment Practices 



Rating Category 


Overall 
(JV= 146) 


CBTG 
(n = 67) 


Educate/ 
Innovate 
( n = 44) 


eLearning 

Project 

(» = 17) 


LNV 
(» = 12) 


No impact 


8.9% 


4.5% 


2.3% 


11.8% 


41.7% 


Slight impact to date 


43.2% 


35.8% 


56.8% 


41.2% 


33.3% 


Noticeable impact 


32.9% 


37.3% 


31.8% 


35.3% 


16.7% 


Very strong impact 


15.1% 


22.4% 


9.1% 


11.8% 


8.3% 



Note. Not all columns total 100.0% due to rounding. The overall column includes three respondents from RWDC and three 
respondents whose programs were not identified. 



The teacher interviews conducted in the four programs clarified the ways in which the eLearning 
Project, the Educate/Innovate Program, and the CBTG Program affected assessment practices, 
and why LNV had little impact in this regard. Teachers in these three programs reported that 
collaborative tools (e.g., Google Docs) allowed students to share their work with teachers more 
frequently. This provided teachers with greater opportunities to provide formative assessments 
(e.g., feedback on a draft of a paper). Students were also engaged more often in self-assessment 
and peer evaluation. As a CBTG teacher explained, 

Without [using] the document cameras to [share student] work [with the rest of the class], 
it would be very difficult to see students go through the process [of] listening to 
suggestions [from the audience], becoming a critical thinker, and using the [feedback] to 
[produce] an improved piece of writing at the end. 

Teachers from the eLearning Project and the CBTG Program have also used polling tools, such 
as Student Response Systems (SRS) and Google Forms, to poll students’ opinions, test their 
understanding, or receive anonymous responses to more sensitive topics. By using SRS devices, 
an eLearning Project teacher explained, “I know immediately what kids [are] struggling with 
[and] can either help them immediately or plan [to address it] tomorrow.” However, these 
programs have not yet influenced summative assessment. LNV teachers explained that the LNV 
resources were supplemental to a unit and did not alter the assessments that teachers would use. 

Overall Impact on Teaching Practice 

In terms of the overall teaching practice, a strong majority (70 percent or more) of the survey 
respondents across all Ed-Tech grant programs reported a noticeable to very strong impact on 
their teaching practice, as shown in Table 31, including their enjoyment of teaching and their 
effectiveness as teachers. Responses varied slightly by grant program, with eLearning Project 
teachers reporting the greatest impacts on their effectiveness as a teacher (82 percent indicated a 
noticeable to very strong impact), and CBTG Program teachers reporting the greatest impacts on 
their enjoyment of teaching (76 percent indicated a noticeable to very strong impact). In contrast 
to the CBTG Program, the Educate/Innovate Program, and the eLearning Project, only one half 
of the LNV teachers reported a noticeable to very strong impact on their effectiveness as teachers 
and their enjoyment of teaching. A complete summary of teachers’ responses to this item can be 
found in Appendix B, Table 5. 
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Table 31. Proportion of Teachers Rating Program Impact as Noticeable or Very Strong 



Overall, how much of an impact has 
your Ed-Tech grant program had on 


Overall 
fiV =146) 


CBTG 
(n = 67) 


Educate/ 
Innovate 
(n = 44) 


eLearning 
Project 
(n = 17) 


LNV 
(n = 12) 


Y our effectiveness as a teacher. 


70 . 5 % 


74.6% 


72.7% 


82.4% 


50.0% 


Your enjoyment of teaching. 


69 . 7 % 


76.1% 


64.7% 


70.6% 


50.0% 



Note. The overall column includes three respondents from RWDC and three respondents whose programs were not identified. 
The Ns vary slightly across rows. 



Summary 

In summary, teachers report that the Ed-Tech grant program had a noticeable to very strong 
impact on instructional and assessment practices, with the exception of LNV teachers. Moreover, 
a large majority of teachers believed that the programs enhanced their enjoyment of teaching and 
their effectiveness as teachers. The programs promoted student-centered instruction in a variety 
of ways. Teachers reported, on average, more frequent assignment of projects. During the 
interviews, teachers described more project-based, collaborative, and personalized learning and 
greater differentiation of instruction. Teachers reported frequent student use of technology for 
several purposes, such as additional practice or skill reinforcement, online research, and paper 
writing. Compared to last year, teachers reported more frequent student use of technology for 
every type of academic purpose. 

Question 4: What Are the Learning Outcomes of the Program in Terms of 
Student Engagement and Motivation and Mastery of Vermont Grade-Level 
Expectations? 

In this subsection, we highlight findings from the teacher surveys that indicate participant 
perceptions of the impact of the program on student learning. 

4a. Student Motivation and Engagement 

Through the survey, teachers were asked to rate the extent to which their participation in their 
school’s Ed-Tech grant program had improved student engagement. Across programs, 78 percent 
of the teachers reported that they observed a noticeable to very strong impact on student 
engagement. These findings are consistent with the findings of the 2009-10 teacher survey, in 
which approximately three fourths of the teachers rated the impact on active engagement in 
lessons as moderate (35 percent) or very much (38 percent). At the individual grant level, 
teachers responding for the CBTG Program reported a higher level of impact on student 
engagement, and LNV teachers reported a lower level of impact, in comparison to the other grant 
programs (see Table 32). 
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Table 32. Impact of Program on Student Engagement and Motivation 

as Indicated by Teachers 



Overall, how much of an impact has your 
Ed-Tech grant program had on your 
students’ level of engagement? 


Overall 
{N = 143) 


CBTG 
(n = 66) 


Educate/ 
Innovate 
(n = 42) 


eLearning 
Project 
(n = 17) 


LNV 
(n = 12) 


No impact/negative impact 


4 . 2 % 


3.0% 


2.4% 


0.0% 


25.0% 


Slight impact to date 


17 . 5 % 


10.6% 


21.4% 


17.6% 


33.3% 


Noticeable impact 


52 . 4 % 


50.0% 


57.1% 


64.7% 


41.7% 


Very strong impact 


25 . 9 % 


36.4% 


19.0% 


17.6% 


0.0% 



Interviews with staff across all grant programs indicated that the integration of technology 
improved student engagement and motivation for learning. Interviewees offered a few reasons 
and examples. Across the three programs that integrated technology into project-based learning, 
(the CBTG Program, the Educate/Innovate Program, and the eLearning Project), interviewees 
reported that the technology element increased student interest in and excitement about their 
projects. As a teacher from the eLearning Project explained, “Whatever [students are doing] with 
technology, they’re engaged. Whether it’s because it’s theirs, they can be on their own with it . . . 
or it’s just something interesting and fascinating for them.” In the LNV program, interviewees 
consistently noted that their students have shown increased engagement and motivation when a 
unit includes LNV and CILC programming. Several interviewees noted that students enjoy the 
opportunity to with an outside expert or with peers in other schools. 

Additionally, interviewees from the CBTG Program, the Educate/Innovate Program, and the 
eLearning Project sites noted that students have also shown improved behavior and increased 
responsibility when they are given access to technology in the classroom and, in some cases, at 
home. 

4b. Impact on Student Skills and Mastery of Grade-Level Expectations 

Respondents to the teacher survey rated the extent to which their participation in the Ed-Tech 
grant program improved student learning outcomes, including conceptual understanding, ability 
to work collaboratively, and performance on standardized tests. These findings are summarized 
in Table 33. A complete summary of responses by item and program is in Appendix B, Table 6. 



Table 33. Impact of Program on Student Skills as Indicated by Teachers 



Overall, how much of an impact has your 
Ed-Tech grant program had on 


N 


No Impact/ 
Negative 
Impact 


Slight 
Impact 
(to Date) 


Noticeable 

Impact 


Very 

Strong 

Impact 


Your students’ understanding of content. 


140 


8.6% 


28.6% 


50.7% 


12.1% 


Your students’ ability to work collaboratively. 


139 


13.7% 


27.3% 


39.6% 


19.4% 


Your students’ perfomiance on standardized 
tests. 


118 


61.0% 


28.8% 


9.3% 


0.8% 



Teachers indicated that the strongest learning outcome was for understanding of content. Across 
all programs, 63 percent of the teachers reported a noticeable (5 1 percent) or very strong 
(12 percent) impact on students’ understanding of content. Teachers with the CBTG Program 
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indicated the greatest impact, with nearly 80 percent reporting a noticeable (60 percent) or very 
strong (19 percent) impact. This finding may reflect the program’s targeting of specific grade- 
level expectations. 

Across programs, 59 percent of the teachers indicated that their Ed-Tech grant programs had a 
noticeable (40 percent) to very strong (19 percent) impact on their students’ ability to work 
collaboratively. These findings were consistent across the grant programs with the exception of 
LNV, where 25 percent of the teachers indicated a noticeable impact (and none indicated a 
strong impact). This finding is consistent with the changes in instructional practice reported in 
regard to Evaluation Question 3, namely, greater assignment of group projects, greater 
opportunities to use collaborative tools (e.g., Google Docs), and additional opportunities for 
students to help one another. 

Teachers were less confident about the impact of the grant on students’ performance on 
standardized tests; 10 percent indicated that it had a noticeable impact (see Appendix B, 

Table 5). 

Through the interviews, teachers from all four grant programs noted that they have anecdotally 
observed that students’ knowledge of content areas and technological skills have improved with 
technology integration. The interviews revealed several reasons why teachers feel that 
technology helps students to better understand academic content. Teachers stated that the 
increases in student-centered instruction, as described previously (e.g., self-pacing and additional 
media for learning), enhance student engagement, content knowledge, and skill development. 

Interviewees from the CBTG Program, the Educate/Innovate Program, and the eLearning Project 
consistently reported that increased student engagement and motivation translates to increased 
comprehension of material. In the words of one CBTG grant manager, 

[Teachers] mentioned that [increased student] engagement [has] translated to 
improve[ed] writing [skills]. [Students] were very interested in doing self-correction 
[when recording their voice with a device]. They could hear their language, and they 
would go back and do it again on their own. They would be reading and then they’d hear 
what they were saying and they’d recognize what they needed [to correct in their speech]. 

Summary 

In summary, teachers strongly perceived that their Ed-Tech grant programs made a positive 
impact on student engagement and motivation. In the interviews, teachers noted several aspects 
of the program that promoted engagement and motivation: the hands-on use of technology, the 
opportunity to connect with experts and peers at other locations, and project-based learning. 
Teachers frequently reported that the programs enhanced students’ understanding of content. 
Teachers stated that the reason for this impact is the greater student engagement in tasks, as well 
as changes in instructional practice that allowed for greater self-pacing and additional media for 
learning. Teachers also typically reported that the program promoted student ability to work 
collaboratively. This finding reflects the finding, reported in a previous subsection, of greater 
assignment of group projects, greater opportunities to use collaborative tools (e.g., Google 
Docs), and additional opportunities for students to help one another. Teachers typically did not 
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believe the grant program made a strong impact on student perfonnance on standardized tests. 
Ratings of impact on student engagement and content understanding were stronger for the CBTG 
Program and weaker for the LNV program. 

Question 5: To What Extent Are Changes in Teaching and Learning 
Adopted and Sustained? 

In addition to the impact and the outcomes of participation in the Ed-Tech grant programs, an 
important question of implementation is the ability of school staff and leadership to sustain the 
effects of the Ed-Tech grant program at their schools in future school years, particularly given 
the short-term nature of many of the grants. Program participants were asked through both the 
survey and the interviews to speak about the sustainability of their programs and their plans to 
continue and expand the use of Ed-Tech grant program practices, as well as plans to maintain or 
procure funding for the program in the future. 

Through the survey, grant managers were asked whether they expect local funding to be 
available for their Ed-Tech grant programs. As Table 34 shows, the majority of the respondents 
(64 percent) reported that they plan to have local funding available to sustain their grant 
programs once grant funding runs out. This compares to 55 percent of the grant managers 
responding to the 2009-10 survey who indicated that they were unsure whether local funding 
would be available. 

Of the 23 grant managers who provided additional information about the local funds they would 
receive, the majority (61 percent) reported that the funds would be used to purchase additional 
hardware and materials. Additionally, 26 percent of the grant managers reported that their 
schools’ budgetary funds would be used to pay for additional training or ongoing professional 
development. Three grant managers also indicated that additional local funding would not be 
provided because it is not needed to continue the grant program. 



Table 34. Frequency of Grant Managers Indicating the Availability of Local Funding 
After Completion of the Ed-Tech Grant Program (N = 39) 



Will local funding be available to continue the 
program once grant funding is finished? 


Number of 
Respondents 


Percentage of Total 
Respondents 


Yes 


25 


64.1% 


No 


14 


35.9% 



Through interviews with teachers, grant managers, and principals, respondents indicated that 
most of the grant programs had some plans to expand the work of the grant to other teachers in 
the school and, in some cases, to teachers in other schools. In general, teachers who participated 
in these grant programs have become “experts” and will help train other teachers on what they 
have learned. Participants in the CBTG Program, the Educate/Innovate Program, and the 
eLearning Project intend to continue integrating technology in the manner they have learned. 
Grant managers across most programs are seeking, or have already secured, funding to provide 
additional training, maintenance of equipment, and technology support after the Ed-Tech grant 
funding runs out. The plans for sustaining funding differ somewhat by program. The findings 
about specific programs are as follows: 
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• eLearning Project. Teachers at all four sites interviewed in spring 2011 stated they will 
continue to use the technology and skills they learned from this project. In some cases, 
part-time positions have been created to provide teachers with a dedicated technology 
integration resource on-site. At each site, there are plans to expand participation in the 
project, drawing on educators both within and beyond the school. For example, one 
school’s principal will also continue to encourage schoolwide technology integration 
beyond the eLearning Project team. In the words of this principal, 

The work that we started over the last year will continue at least [into] the next three 
years until we get to a point where we have trained everybody in the building [to have] a 
core understanding of curriculum redesign. 

Moreover, each of the eLearning Project schools is committed to serving as a model 
technology integration school that other educators may visit. 

• CBTG Program. The CBTG sites typically involved a subset of teachers focused in a 
particular curriculum area. Teachers typically stated they plan to share their knowledge 
and experience with other teachers in the school. As a principal of a CBTG school noted, 
“A lot of [what we have piloted] has end up spreading across within our grades, within 
the three or the five student classrooms, but also in the lower grades too.” CBTG schools 
have established plans for sustaining the program once grant funding ends. As an 
interviewee explained, even with impending budget cuts, her school is committed to 
supporting technology integration through internal and external resources. She noted, 

We’re not expecting to fund this all ourselves, but we think that we can continue to 
access outside resources that will help us to do that. ... We’re going through a period of 
budget cuts here as every school in Vennont is. But one area that we are keeping intact is 
our technology commitment. 

• Educate/Innovate Program. Like the CBTG Program, the Educate/Innovate Program 
involves a subset of teachers from one or several schools. These teachers typically 
reported, like their CBTG counterparts, that they plan to share their knowledge and 
experiences with other teachers both within their schools and, in some cases, beyond their 
schools. At three of the four Educate/Innovate sites, grant managers reported plans to find 
additional resources for continued professional development of teachers, including the 
engagement of grantee teachers as lead trainers of their colleagues. One site that piloted 
several different technological devices will detennine in which devices the school will 
choose to invest, based on teachers’ experiences with implementation. The grant manager 
from this site explained, 

The intent [of this pilot project] was to figure out what devices would be powerful [and] 
effective [tools for enhancing student learning]. If [one or two are] really that powerful, 
then we’ll figure out a way to implement it [on a wider scale]. 

• LNV. Interviewees noted that at least one additional teacher in each grantee school plans 
to use LNV and CILC programs in the spring semester. However, some teachers are less 
inclined to use the LNV system because they are not comfortable with using this type of 
new technology, have limited time to leam how to use it, are resistant to alter their 
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current teaching practices, or are focused on other projects with students. LNV teachers 
at several schools noted that sustaining the cost of CILC programming through the school 
budget was not feasible given the current state of budget cuts. Schools that have acquired 
LNV videoconferencing technology are planning for continued use of this equipment as a 
classroom-to-classroom collaboration tool, which does not cost schools money. A 
minority of schools is actively looking to secure funding to cover the cost of credits when 
free credits are no longer available. However, all LNV interviewees stated that there was 
not strong interest among other teachers in their schools in adopting the 
videoconferencing technology due to the investment of time in learning to use it. Thus, 
the sustainability of the LNV program faces challenges of cost and educator interest. 

Summary 

Responses to the grant manager survey indicate that nearly two thirds of the grantees have plans 
to sustain their programs with additional funds for the purchase of equipment, software, and 
additional or ongoing training. Participants in the CBTG Program, the Educate/Innovate 
Program, and the eLearning Project intend to continue integrating technology in the manner they 
have learned and to expand the work of the grant to other educators (both within and beyond 
their schools). LNV sites expect to continue to use their videoconferencing equipment as a 
classroom-to-classroom collaboration tool, which does not cost schools money. LNV staff 
members were uncertain about expanding the use of this equipment to other educators in the 
school due to the large investment of time required to learn how to use the technology. At several 
schools, the cost of CILC programming was seen as a barrier to continued use of that resource. 

Question 6: To What Extent Did the Program Support the Development 
and the Deployment of Online Learning Opportunities? 

The sixth and final evaluation question pertains to the VTVLC program, which offers online 
courses to students in a wide variety of subject areas. Schools receive seats for their students in 
courses being offered through VTVLC in exchange for providing a teacher to teach an online 
course. VTVLC also offers professional development for teachers, guidance counselors, and 
administrators on topics that involve online education and learning. The VTVLC program 
launched in the summer of 2010 across 14 schools, with 75 students enrolled in full-credit math, 
science, social studies, or English language arts courses. In fall 2010, 143 students across 19 
schools enrolled in VTVLC online courses, which were expanded to include art and technology 
courses. The program expanded to 33 schools and a total of 60 trained online teachers in the 
2010-11 academic year. 

The program served partner high schools and nonpartner high schools and middle schools. 
Partner schools are those that provide financial contributions to the program to help cover its 
administrative costs and provide at least one teacher as a VTVLC instructor. Students at partner 
schools may enroll in courses without incurring added tuition fees. Students from nonpartner 
schools (both in state and out of state) may enroll in VTVLC but pay a tuition fee. A summer 
credit recovery option is also available to students for tuition fees. To address Evaluation 
Question 6, it was necessary to articulate five more specific evaluation questions, around which 
this section of the report is organized: 
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1 . To what extent are students enrolling in and passing VTVLC courses? How do these rates 
vary by curriculum area and school? 

2. For what reasons do students enroll in VTVLC courses? 

3. What is the impact of school participation in VTVLC on the availability of courses not 
offered by a local school, the accessibility of courses to reduce scheduling conflicts, and 
the flexibility of learning opportunities? 

4. How satisfied are school leaders, personnel, and students with VTVLC courses, 
administrative support, and professional development? 

5. How sustainable is the VTVLC program, and what are the barriers to participation? 

Data Collection 

Here we present the findings from the spring 2011 principal survey, the partner school extant 
data analysis, and the fall 2010 site visits to four schools participating in VTVLC. During these 
site visits, the evaluator conducted interviews with teachers, guidance counselors, and principals 
and focus groups with students. Although the site visits were described at the beginning of this 
report, the additional data collection methods that pertain only to VTVLC are described here. 

Principal Survey. The evaluation team administered a principal survey in spring 2011. The 
mixed format survey consisted of 12 multiple choice, rating scale, and open-ended items. The 
survey was roughly 5 minutes in length. In spring 2011, VTVLC provided AIR with a list of 
28 participating principals, from partner and nonpartner schools, and their e-mail addresses. All 
28 principals were invited by e-mail to take the 2011 survey. The survey was administered 
online, and a total of 15 principals (54 percent) completed it. A copy of the principal survey is 
included in Appendix A. 

Extant Data. The VTVLC program provided the evaluation team with student-level enrollment 
and perfonnance data. Data fields contained school, course, student number, final grade, 
academic tenn, and course completion. These data represent a total of 143 enrollments in 
21 courses across 17 partner schools, including 1 home school. One school was a vocational and 
technical school that hosted the VTVLC program, serving students from many surrounding high 
schools. 

Interviews. The evaluation team visited four schools participating in VTVLC during fall 2010, 
consisting of three partner schools and one nonpartner school. The findings from these 
interviews were included in the spring 2011 interim report, and they are briefly summarized here 
as well. 

Enrollment, Completion, and Student Performance 

Overall, there were 143 enrollments in VTVLC courses by students in partner schools in 
2010-11. Each enrollment represented a unique student (i.e., no student enrolled in multiple 
courses). Of these enrollments, 86 percent completed their courses, with an average final grade 
of 82 percent. 
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Enrollment and Completion by Subject Area. Course enrollment and completion were 
analyzed by subject area. Foreign language had the highest enrollment with 34 students, and 
English language arts had the next highest with 23 students. Over 76 percent of the students 
finished their coursework with a passing grade in each subject area. Science and health and 
fitness courses had the highest completion rates (100 percent), and foreign language courses had 
the lowest (77 percent). These results are presented in Figure 5. 

Figure 5. VTVLC Enrollment and Completion by Subject Area, 2010-11 




Art ELA Health/Fitness Foreign Math Science Social Studies Technology 

Language 



As noted, across all subject areas, the average final course grade was 82 percent. Subject areas 
varied little with respect to final grades, as depicted in Figure 6. Courses in English language arts 
had the lowest average student grade (73 percent), while courses in art had the highest 
(87 percent). 

Figure 6. VTVLC Partner School Average Final Grade by Subject Area, 

2010-11 Academic Year 
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Enrollment and Completion by School. Course performance and completion rates were 
analyzed by school. Cabot School had the highest enrollment with 22 students, and Lyndon 



American Institutes for Research 



2010-11 Final Report on Vermont’s Ed-Tech Program — 55 





Institute and River Valley Technical Center had the lowest, with only 1 student enrolled at each 
location. One home-schooled student was also enrolled in the VTVLC program. As noted, 
completion rates were strong across most schools, with an average of 86 percent of students 
completing their courses. Several schools had a 100 percent completion rate. North Country 
Union High School and Woodstock Union High School had the lowest completion rates (37.5 
percent and 50 percent, respectively). Spaulding High School had the highest average final grade 
(93.1 percent), and Lyndon Institute had the lowest (72 percent). See Table 35. 



Table 35. Total Enrollment and Completion in VTVLC by Partner School, 2010-11 



School 


Number Enrolled 


Percent Complete 


Average Grade 


Bellows Free Academy, St. Adams 


2 


100.0% 


73.5% 


Cabot School 


22 


90.9% 


73.6% 


Green Mountain Union High School 


18 


100.0% 


79.8% 


Hartford High School 


7 


100.0% 


92.9% 


Home School 


1 


100.0% 


77.0% 


Lyndon Institute 


1 


100.0% 


72.0% 


Milton High School 


4 


100.0% 


90.5% 


Montpelier High School 


3 


100.0% 


84.7% 


Mount Mansfield Union High School 


12 


83.3% 


87.0% 


North Country Union High School 


8 


37.5% 


75.0% 


Peoples Academy 


16 


75.0% 


76.9% 


River Valley Technical Center 


1 


100.0% 


88.0% 


South Burlington High School 


14 


92.9% 


78.1% 


Spaulding High School 


11 


100.0% 


93.1% 


Springfield High School 


8 


50.0% 


78.3% 


U-32 High School 


11 


100.0% 


87.9% 


Woodstock Union High School 


4 


50.0% 


78.0% 


Total 


143 


86.0% 


81.6% 



Reasons for Participation 

Principals and guidance counselors from each school mentioned two major factors that motivated 
students to participate in VTVLC. First, students were enrolling in VTVLC courses to obtain 
credit in courses they were interested in but were not offered at their schools. These courses 
ranged from Advanced Placement courses in science and mathematics to advanced art courses in 
photography and digital animation. Second, students were enrolling in VTVLC courses to 
recover credit. At one school, students who had lost credit were targeted for enrollment in 
VTVLC. As one guidance counselor stated, “[VTVLC] gives a student hope that okay, I screwed 
up, I don’t have to be punished for a whole year or maybe my whole career for that matter.” 

Impact on Course Availability, Accessibility, and Flexibility 

A total of 13 out of the 15 respondents answered the items on the spring 2011 principal survey 
about program impact. Figure 7 presents these results. A majority of the respondents perceived a 
noticeable or very strong program impact on students’ opportunities to enroll in courses not 
otherwise available. This finding is consistent with the fall 2010 site visits, during which staff at 
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all 4 schools reported that students enrolled in VTVLC to gain access to courses not offered at 
their schools. A minority of 3 to 4 principals perceived a noticeable or strong impact on the 
flexibility of students’ schedules, opportunities to enroll in advanced coursework, and the 
school’s scheduling flexibility. 



Figure 7. Principal Ratings of VTVLC Impact on School (N = 13) 
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Principals also responded to an item asking them how their students had benefited from 
participating in VTVLC. Figure 8 presents those results. At least 8 of the 13 principals agreed 
that their students learned to work independently, to work in an online community, and personal 
responsibility. 

Figure 8, Number of Principals Identifying Different Benefits to Students ( N = 12) 
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In summary, a majority of the principals reported that VTVLC made a noticeable or strong 
impact on expanding course availability (i.e., enrollment in courses that would not have been 
otherwise available). A minority reported that VTVLC made a noticeable or strong impact on the 
flexibility of students’ schedules, opportunities to enroll in advanced coursework, and the 



American Institutes for Research 



2010-11 Final Report on Vermont’s Ed-Tech Program — 57 






school’s scheduling flexibility. Most of the principals agreed that the students who did 
participate in the program learned several skills, such as working independently, working in an 
online community, and taking personal responsibility over learning. 

Program Satisfaction 

Principals were asked to indicate their level of satisfaction with the VTVLC program in the 2011 
survey. Figure 9 presents these results. Ten of the 13 respondents were moderately (6) or very (4) 
satisfied with the program. 



Figure 9. Principal Satisfaction with VTVLC, Spring 2011 Principal Survey (N = 13) 
Is n I 
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Professional Development. VTVLC offers a series of four professional development courses 
designed to prepare teachers in the practice of online course facilitation and prepare them for the 
technical and administrative functions of the online course. Interviewees from each school found 
the professional development courses to be useful for navigating the learning management 
system (Moodle). However, teachers from two schools reported that they were not fully prepared 
to administer an online course. The transfer of typical classroom operating procedures to the 
online environment, such as keeping a grade book up-to-date, was a common problem. 

Administrative Procedures. Administrative procedures refer to the management of the online 
program by school staff, particularly the interface with VTVLC processes for enrollment and 
grade reporting. Principal, guidance counselor, and teacher interviewees from each school 
mentioned that that they were happy with the procedures used to enroll students in the fall. 
Moreover, school-based staff typically stated that the VTVLC administrators communicated 
effectively and responded promptly to questions about administrative procedures. 

Quality of Online Courses 

Teachers and students provided their assessment of course quality in terms of the curricular 
focus, instructional fonnats, assessment procedures, and the overall quality of student work. 

Curriculum. Teachers and students at each school provided their feedback on the quality of the 
online courses they were teaching and taking with respect to the curriculum. Students and 
teachers at each school felt that their courses were equipped with a curriculum with good depth 
and breadth of content coverage. One calculus teacher explained, 
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As for the content, I like it. I like how it’s not all multiple choice; there are some writing 
assignments. So, it gives a much better view of the student’s understanding than just 
multiple guess. It’s much more thorough than courses we’ve used in the past. I do think 
they’re going into more detail. 

Another student said, “Our course is quite balanced.” One teacher did, however, mention that 
there was some redundancy in the course content, but it was otherwise satisfactory. 

Instructional Formats. Students and teachers also provided feedback on course structure and 
instructional strategies. A common theme across all schools was that online communication 
between peers was limited. Dissatisfied with the current course structure, one teacher explained, 

I don’t think it utilizes the forums very well. So, all the students are really independent in 
the course right now, which is ok, they’re all doing their own work, but they might not 
even be aware that there are other students.... I don’t think the forum has been used very 
well. 

Teachers were aware of the online forums but were perhaps not using them regularly because of 
the time commitment necessary to leam to navigate the basic online platform. One teacher said, 
“I do want to incorporate more of the online discussions, but I’m not working that in until we 
were more comfortable.” Students also mentioned that peer-to-peer interactions were rare except 
in situations in which multiple students from a single school were enrolled in a course. In this 
scenario, students tended to work together in person when possible. 

Barriers and Challenges 

Principals who responded to the 2011 survey were asked to select the different barriers to their 
school’s continued participation in VTVLC. Respondents indicated that low student interest and 
program cost were potential barriers (see Figure 10). 

Figure 10. Principal Responses to Barriers to Participation in VTVLC (N = 13) 
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They also had the option to provide other barriers and challenges not listed in the survey. Five 
principals submitted the following responses, each addressing a different barrier: 

• There was a lack of teacher availability. 

• There was a lack of teacher preparation and student readiness. 

• It is hard to compete with the range of classes offered at [online provider]. 

• VTVLC is not conducive to block scheduling. 

• There were technical difficulties, including problems with connectivity. 

Sustainability 

In the 2011 survey, principals responded to one question about the likelihood of continuing their 
school’s participation in the VTVLC program. Figure 1 1 presents these results. Of the 13 
principals who responded to this item, 1 1 said they would probably or certainly continue. 

Figure 11. Principal Ratings of Likelihood of Continued Participation in VTVLC (N = 13) 




We certainly We probably Unsure if we We probably We certainly 
will not will not will or will will will 

participate participate not participate participate 

participate 



Principals were asked to identify any ways that VTVLC could be improved. Based on feedback 
from principals, two themes emerged. The first theme, submitted by three principals, relates to 
communication, including more orientation for students, teachers, and staff. The second theme 
relates to online peer collaboration. Two principals suggested that VTVLC should increase the 
frequency of online collaboration and online forum usage. 

Summary 

• To what extent are students enrolling in and passing VTVLC courses? How do these 
rates vary by curriculum area and school? Students who enrolled in VTVLC tended to 
complete their coursework. Of the 143 students enrolled in the VTVLC program, 

86 percent completed their courses with passing grades. Furthermore, the average final 
grade across all VTVLC courses was 82 percent. 

• For what reasons did students enroll in VTVLC courses? According to principals and 
guidance counselors, students were using VTVLC to enroll in courses that were 
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otherwise not available to them. In addition, students in need of credit recovery were 
encouraged to participate in VTVLC. 

• What is the impact of school participation in VTVLC on the availability of courses 
not offered by a local school, the accessibility of courses to reduce scheduling 
conflicts, and the flexibility of learning opportunities? Most principal survey 
respondents indicated at least a slight impact of the VTVLC program on course 
accessibility, scheduling, and the flexibility of learning opportunities. When considering 
communication and the quality of the online courses, most principals who responded to 
the 2011 survey suggested that VTVLC was as at least comparable in effectiveness to 
similar programs. 

• How satisfied are school leaders, personnel, and students with VTVLC overall, 
course quality, administrative support, and its professional development? Most 
principals were moderately or very satisfied with VTVLC as a whole. School staff stated 
that VTVLC administrative support was effective and appreciated the responsiveness of 
VTVLC staff to administrative questions. Principals also expressed an interest in greater 
course variety. Regarding professional development, Interviewees at each school stated 
that teacher training was effective overall, but some teachers identified certain technical 
skills for which they were not fully prepared. Teachers and students were pleased with 
the depth and breadth of the curriculum. However, teachers and students generally agreed 
that there was little opportunity for peer-to-peer interaction among students. 

• How sustainable is the VTVLC program, and what are the barriers to 
participation? Eleven of the 13 principal respondents suggested that they would 
probably or certainly continue their school’s participation in VTVLC. 
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Section V. eLearning Project Spotlight Study 



The purpose of this Spotlight Study is to depict the fidelity of the implementation of the 
eLearning Project in relation to its overall project plan, and to describe the outcomes of the 
program in relation to its goals. This study was undertaken in light of the large investment of the 
VTDOE in the eLearning Project. The overall goals for this project were for educators and 
school leaders to: 

1) become more proficient in student-centered, technology-rich teaching and learning, 
supported by research-based professional development; 

2) contribute to an interactive network of resources to support student-centered, technology- 
rich teaching and learning; and 

3) engage the community, and build a community of learning, around 21 st Century schools 
and technology integration. 

The project was based in six demonstration sites. At each site, there was a core project team 
consisting of five to seven teachers, the school principal, and a technology specialist. This team 
worked with an external coach (paid for and assigned by the overall project leadership) to 
accomplish a specific scope of work. Each project site developed its own scope of work, which 
was referred to as a syllabus. Each project team was expected to develop resources for 
technology integration, and to share these resources via an interactive website (eleamingvt.org). 
Each site was also expected to plan and host three regional workshops open to teachers in their 
own school and neighboring schools. Finally, each site was expected to develop plans to sustain 
and expand its efforts, in the school and in its region. Throughout these steps, each team was 
expected to receive research-based professional development from its coach. 

This study explored fidelity of implementation by conducting interviews at four of the six 
eLearning sites. These sites were purposively sampled to reflect geographic diversity. One of the 
sites was an elementary school, one was a combined elementary/middle school, and two were 
middle schools. As described in the Evaluation Design and Methods section, during spring 2011, 
the evaluation team conducted a total of 13 interviews of members of each site's eLearning team 
(teachers and the school principal) and each team's eLearning coach (assigned by and paid for by 
the program). However, teachers from one school did not respond to the interview request; 
therefore findings from that site are based on principal and coach interviews. In addition to these 
interviews, evaluators reviewed artifacts of project work that each team posted to their project 
websites (sub-sites of elearningvt.org). 

Fidelity of Program Implementation 

This section summarizes the extent to which the team structure and planned work at each site 
reflected fidelity to the design of the eLearning Project. 
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Team Structure 



The four eLearning teams comprised five to seven members, including the school principal, a 
library and/or technology specialist, teachers, and other school or district technology support 
staff. Each team was paired with an external coach paid for by the grant and hired by the grant 
manager. In interviews, these coaches all described themselves as educators with experience in 
technology integration. Therefore, the composition of the eLearning teams is consistent with the 
number and type of individuals expected to join the teams. 

Work Plans 

Each team was expected to develop a customized syllabus to outline the team’s work on the 
project. The assignments on the syllabus were expected to be in alignment with the goals of the 
project. 

All four teams developed a customized, site-based syllabus and publicized this information on 
their team website. Each team developed this syllabus collaboratively, working both in-person 
and online. One coach explained, 

[The team developed the course syllabus] by meeting together [to] integrate [this work] 
into work they had already done and [plan ways to modify their existing] curriculum to 
deliver [content] differently [now] that [each student has] access to [a] laptop. [The 
faculty] were trying to tie [the] Marzano principles [into the syllabus as well]. 

Three teams posted a working syllabus document on their team’s website. The fourth eLearning 
team did not develop a traditional syllabus document, but set up their team website as a map of 
their goals, strategies, activities, and resources. These components are similar to those included 
on the more traditional, document-based eLearning syllabi. Syllabi typically included a course 
description, learning objectives, assignments, a timeline, and a list of readings and resources. 
One syllabus was more detailed; it also included a list of technology and education standards 
addressed in the course. Thus, it appears that all teams developed a syllabus or a syllabus-like 
document, although these documents varied in their format and level of detail. 

Overall, these work plans reflected the purpose of the eLearning Project. Interviewees were 
asked to describe the main purpose or motivation for their team’s project. Interviewees from all 
four schools described the main purposes of their project in a manner consistent with the overall 
project goals. The common themes were as follows: 1) to promote school-wide use of 
technology integration in the classroom in a meaningful and engaging way; 2) to provide 
professional development for teachers; and 3) to become a model school from which others may 
visit and learn. As a principal explained, 

[The purpose of the eLearning grant is] to promote technology use school wide with the 
intent to have [our school] be[come] a model school [that] others could come and visit 
and see technology being used well. 

It should be noted that three of the four teams focused on supporting one-to-one computing 
initiatives that recently started at their schools. In summary, the overall purposes of the projects 
were consistent with the stated goals of the eLearning Project. 
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Completion of work plans. Interviews and review of artifacts indicated that, for the most part, 
teams accomplished the work stated in their syllabi. At all four schools, team members were 
assigned the task of planning, implementing and evaluating technology-rich projects to use in 
their classroom with students. As an end work product, teachers developed teaching pedagogies, 
lesson plans, rubrics, and assessments for technology-rich projects. According to interviews, at 
two schools, the work of the eLearning team supported the implementation of the one-to-one 
computing initiative. Describing the work accomplished by her team, one coach stated, 

Every grade level team created a wonderful lesson plan infused with technology and 
interdisciplinary content. [They] wrote up [the lesson plan], taught it, [and] provided 
examples of student work [on] the website. We also read a book together and had some 
book discussions on that, but mostly our evidence was based around teachers actually 
being able to create these lessons, teach them and have students produce products based 
on what they taught. 

Example work products and evidence of student learning were posted on eLearning team 
websites to varying degrees of detail. The following summary reflects the outputs of the 
collaboration: 

• One school posted original lesson plans by grade level that include examples of 
assessment tools, standards addressed, and student work products (typically video clips). 

• Another school posted examples of student work products and infonnation on 
technology-integrated teaching pedagogies with links to example lesson plans developed 
by a professional association. 

• A third school posted examples of student work, li nk s to classroom wikis, and research- 
based strategies that teachers should consider when implementing a technology 
integration initiative. 

• A fourth school used its team website to present the school’s one to one computing 
initiative, including logistics and preparation strategies. This school included a few video 
clips of student and teacher reflections, examples of a class project, and links to 
assessment framework resources. 

One instance in which the evaluation team did not find evidence that the planned work was 
implemented was one team’s plan to design a variety of assessments for technology integration. 
Evidence of these assessment tools was not posted on the team’s website. In conclusion, there is 
strong evidence that each team accomplished its assignments related to developing technology- 
rich projects. 

Professional Development 

The project plans described a model of professional development that would support all aspects 
of the work of each team. Analysis of data collected through interviews and a review of 
documents shows that all four eLearning teams adhered to the intended training model of the 
grant program. At the start of the program in January 2010, coaches and eLearning team 
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members participated in a three-day iLEAP Institute. According to three coaches, the training 
provided ideas and resources to use with their school. These included examples, model 
programs, and tools for technology integration into education. Although team members were not 
asked directly for their opinion about the Institute, participants from one team mentioned that 
they benefitted from having time at the Institute for team planning. 

Two coaches acknowledged that there was no training specifically for coaches, with one coach 
noting that limited training of and interaction among coaches was a program weakness. This 
coach explained, “The coaches never met together to really talk and be trained about exactly 
what we should be doing or to even share ideas. It would've been wonderful to share ideas. ” 

In interviews, the team members and their respective coaches described how coaches support 
each team’s project. Based on these interviews, it appears each coach visited his or her eLearning 
school for a total of three full days per month, as anticipated by the program plan. During visits 
to the school, all coaches met with eLearning team members both individually and as a group. 
Three of the coaches also worked with all other teachers in the school, individually or as grade 
level teams. All coaches helped teachers set up and use hardware and software. They also 
addressed teachers’ specific needs for technology integration, such as augmenting existing 
curriculum, aligning curriculum to technology standards, and troubleshooting equipment or other 
issues. Two coaches built Google sites as a forum for team blogs and resources and one coach 
helped teachers better use technology resources at their school. All teams described their coach 
as being a sounding board for questions and responsive to their needs. In the words of one 
teacher, 



[Our coach] knows what [tools and programs] are out there and [s/he] tells us about 
[them], [Otherwise,] I [wouldn’t] have the time to look for it all. [S/he's] a great resource. 
[S/he has a wealth of] knowledge, knowing what is out there, what is available, and what 
is appropriate for the things we wanted to do. 

Team members in three schools stated that their coach was an effective resource. In one school, 
team members felt that the coach role was less critical because of the principal’s strong 
leadership for the program. 

In summary, participation in the I-LEAP institute and site -based coaching occurred in the 
manner anticipated by the project’s plan. Team members generally perceived their coaches as 
helpful, although the specific role of the coach appeared to vary by site. Some coaches noted the 
lack of prior project-specific training for their roles, and a lack of opportunities to collaborate 
with one another. 

Program Outcomes 

This section describes the outcomes of the program in relation to its three stated goals. 

Goal 1: Enhancing educator Practice and Leadership 

In alignment with Goal 1, all coaches and teachers interviewed stated that teachers have adopted 
new instructional practices over the course of the grant. All teachers interviewed indicated that 



American Institutes for Research 



2010-11 Final Report on Vermont’s Ed-Tech Program — 65 




they have developed and implemented technology-rich lesson plans that are appropriate for their 
grade level and content area. Coaches have also observed that teachers and students are using 
Web-based resources and technological devices (i.e. iPad, cameras, etc.) in the classroom on a 
more frequent and consistent basis. One coach, describing the technology tools that teachers 
recently adopted, said as follows: 

Wikis, global presentations, K word quizzes, [various] Google [Apps], Web 2.0 tools, 
voice thread, garage band, [and] flip cameras, etcetera. Basically [teachers have] really 
increased their use of technology and tools as alternative ways of teaching and 
assessment. 

Teachers were asked specifically about how this grant affected their assessment practices. 
Teachers from three schools described how the greater use of collaborative applications (e.g., 
email and Google documents) has increased their use of formative assessment (namely, feedback 
on student's work that is intended to improve the quality of the final product). (This question 
could not be answered in the fourth school because no teachers responded to the interview 
request). Improved opportunities for students to collaborate allow teachers to give students 
feedback more quickly and students to leam from each other and self-correct more often. One 
teacher explained, 

[The other sixth grade teacher and I developed a] website that [is] very student friendly 
[and is a tool that students] use in classes. [The website] has been a great tool for them, 
and for me, so I can keep track of what they're working on. 

Principal leadership. Goal 1 of the project envisions an impact on school leadership in support 
of technology integration. Teachers and coaches from three schools, along with the coach at the 
fourth school, all noted that the principal has been supportive and encouraging of this grant 
program and provided strong leadership for building a school culture of technology integration. 
Principals also provided resources that teachers needed, such as release time and substitute 
teachers. One coach stated, 

[Our principal’s] leadership has been essential. [S/he's] very engaged and involved with 
all [teachers’ goal setting] and making sure that [their goals] were accomplished. S/he 
made sure [that] faculty attend[ed team] meetings, that they were notified in a timely 
way, [and were aware of his] expectations [of what teachers needed] to do. 

Principals from three schools noted that they often met with coaches to discuss teachers’ needs, a 
plan of action, and ways they both can support teachers. Interviewees from the fourth school did 
not discuss specific interactions between the principal and the coach. 

Sustainability. All teams were expected to make plans to sustain their projects. Team members 
from all four schools stated they will continue to use the technology and skills they learned from 
this project. In the words of one teacher, “I think definitely the Google form will continue. I 
think that's been a great resource. ” 

Another aspect of sustainability is to invite additional educators at the school or from other 
schools to join the work of the project. In this regard, at two schools (both with one-to-one laptop 
initiatives), the coaches and principals stated that their entire school staff is involved in this 
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initiative. At the two other sites, the eLearning teams plan to expand the scope of participation in 
various ways. At one school, participating teachers plan to expand technology integration to 
other content areas that they teach. This school’s principal will also continue to encourage 
school-wide technology integration beyond the eLearning team. In the words of the principal, 

The work that we started over the last year will continue at least [into] the next three 
years until we get to a point where we have trained everybody in the building [to have] a 
core understanding of curriculum redesign. 

At the other school, the team plans to expand the program to other teachers, such as those who 
teach elective courses and para-educators. 

Goal 2: Sharing Resources with Other Educators 

Goal 2 of the eLearning Project is for each site to share resources with other educators in their 
school and in their region. The main way that all four eLearning teams disseminated their work 
to others (within the school and outside) was through eLearning team websites (eleamingvt.org). 
Each website was linked to the school’s main website as well as the website of the larger 
eLearning Project. A review of documents posted on eLearning team websites, described in 
detail above (under Work Plans), shows that the types of resources varied greatly by team. 
Whereas some teams provided lesson plans or li nk s to lesson plans, other teams posted student 
work samples, and another team described the logistics of rolling out a one-to-one computing 
initiative. 

Continued use of project website. Team members from three schools said that teachers will 
continue to add to the list of resources and information posted on the website. However, the 
principal from one school commented that teachers may no longer add to their team’s eLearning 
website content because materials were based on a completed school wide project. 

Goal 3: Building a Collaborative Learning Community 

Goal 3 of the eLearning Project is to build a collaborative learning community focused on 
technology integration. In alignment with this goal, within all four teams, team members 
collaborated with each other and other grade level or department teachers on technology 
integration. This collaboration occurred in person and through online communication and sharing 
of documents. Three teams have increased student and teacher collaboration through digital tools 
such as email, class wikis, chat forums, and Google Docs. All four teams have engaged parents, 
community members and other educators by posting numerous materials, resources, and 
examples of student work on their website. Thus, it appears that each team has made progress 
towards this overall project goal. 

Another way that all four teams promoted a community of learners was through presentations at 
technology and/or content-focused conferences. All eLearning teams presented their program 
summary and findings as part of the eLearning Track during the statewide Dynamic Landscapes 
technology conference in May 2011. 

School visits. All four teams noted their schools are now open to having other schools visit and 
leam about their technology initiatives. As noted by a principal, 
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[Allowing visits by other schools is] part of our commitment to the Tarrant Institute. [In] 
three years time [we plan to be] positioned to be a model school in the use of technology 
and its use for leveraging learning. [Even though we are still] working to that [goal], we 
[have] open doors. 

All teams have invited educators to visit their school. They have extended this invitation via their 
websites, through word of mouth, and at conference presentations. Three of the four schools 
have already had teachers from other schools visit and observe the implementation of their one- 
to-one computing initiative. 

Coach-Facilitated Workshops. As a final strategy for sharing resources with educators, and one 
that was stipulated in the grant proposal, each coach facilitated three one-day workshops that 
were open and advertised to educators in the school's region of the state. 5 The number of 
attendees and their affiliation (with the school or from outside of the school) varied at 
workshops. The number of attendees ranged from seven to 30 people. Two coaches said that the 
majority of their attendees were teachers from within the school and only a few were community 
members or professionals from other schools. Two other coaches said that between 75 percent to 
all of attendees were from outside the school and mainly teachers, technology specialists and 
administrators from their school district and beyond. One of the workshops was a hands-on 
demonstration of teachers and students collecting community level data using GPS equipment 
for an eLearning project. Workshops facilitated by three other grantees were more of a 
traditional format. One grantee’s workshops focused mainly on teaching the use of tools 
commonly used by the eLearning team, such as Google Apps, digital storytelling and podcasting. 
Two of the workshops presented a combination of tools used and strategies, challenges, and 
lessons learned from their one to one computing initiatives. 

In summary, all four teams have addressed the project goal of building a collaborative 
community focused on technology integration. They have accomplished this goal primarily by 
increasing collaboration among educators within their school. They have also reached out to 
educators outside their school by presenting at conferences, inviting educators to visit their 
school, and offering coach-facilitated workshops to a regional audience. It appears that the 
number and diversity of educators who benefited from this sharing differed by team; some but 
not all teams had educators external to the school attend their workshops and observe the 
implementation of their projects. 

Conclusions and Recommendations 

The purpose of this study was to evaluate the fidelity of implementation of the eLearning Project 
in relation to its project plan. Implementation was observed across four project teams. Overall, 
the implementation across these sites reflected the goals and plans of the project. All teams 
developed a syllabus or a syllabus-like document, although these documents varied considerably 
in their fonnat and level of detail. All teams completed the scope of work outlined in their 
syllabi, with the exception of one assignment of one team related to designing assessments. 
Participation in the I-LEAP institute and site -based coaching occurred in the manner anticipated 
by the project’s plan. Team members generally perceived their coaches as helpful, although the 



5 This strategy also aligns with the goal of community engagement (goal 3). 
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specific role of the coach appeared to vary by site. Some coaches noted that the project design 
did not include training for coaches or opportunities for coaches to collaborate with one another. 

The teams appeared to have made progress on the three goals of the project. 

• Goal 1: Enhancing educator practice and leadership. All teams reported an impact on 
educator and school leaders practice. Team members report the project has fostered 
technology-rich teaching and learning, and state that the impact is evident in their lesson 
plans and in student use of technology. Principals have assumed an active leadership role; 
their active participation is seen as crucial to the success of the projects. Team members 
across sites expect to continue to use the technology they learned from this project. They 
also have either brought in the other teachers in their school to the project, or plan on 
doing so in the future. 

• Goal 2: Sharing resources with other educators. All teams have used their project 
website to share resources for technology integration with other educators. Whereas some 
teams provided lesson plans or li nk s to lesson plans, other teams posted student work 
samples, and another team described the logistics of rolling out a one-to-one computing 
initiative. In three of four schools, team members expected to continue to share resources 
from their project via their website 

• Goal 3: Building a collaborative learning community. Each team has made progress 
towards creating a collaborative learning community, primarily by enhancing 
collaboration among educators within their school. Teams have shared resources during 
conference presentations and through workshops open to a larger community of 
educators. The breadth of participation in these workshops varied greatly by program, 
however. 

Recommendations 

Based on these findings, three recommendations about the eLearning Project are warranted: 

1. Expanding to additional sites. The eLearning Project appears to be effective in fostering 
a sustainable impact on educator practice and leadership. There are several features of the 
project that appear to be critical to its success. At each site, there was a core planning 
team that developed a specific scope of work and identified specific outcomes to be 
achieved. At several sites, the scope of work was closely aligned with other concurrent 
initiatives of the school. The work of the team was supported by the ongoing involvement 
of the school’s principal and frequent visits by a technology integration coach. Teachers 
at these sites supported each other through extensive collaboration. The VTDOE may 
consider expanding this model to other sites. 

2. Supporting regional networks. Although there was extensive collaboration among 
educators within sites, there was less progress with expanding this community to include 
educators in the region. It was not clear why some sites were more successful than others 
in drawing in outside educators. If this result is of importance to VTDOE, it ought to 



American Institutes for Research 



2010-11 Final Report on Vermont’s Ed-Tech Program — 69 




consider providing greater guidance and resources for reaching out to the larger 
community (of both educators and other citizens). 

3. Enabling collaboration among coaches. Should a similar project be offered in the 
future, VTDOE may consider allotting time for coaches to collaborate and seek advice 
and support from each other. Coaches indicated that opportunities to collaborate would 
have been useful for their efforts to support their teams. 
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VI. Conclusions and Recommendations 



The following represent the conclusions and recommendations from the evaluation conducted by 
AIR on the Ed-Tech Program, with a focus on discretionary grants which included the following 
programs: the CBTG Program, the Educate/Innovate Program, VTVLC, LNV, the eLearning 
Project, and RWDC. The conclusions are summarized by the six evaluation questions. 

Question 1: To What Extent and With What Fidelity Are the Grantees 
Making Progress Toward Their Objectives? 

Overall, grantees have made significant progress toward their objectives and are largely 
implementing their programs with fidelity. This is observed in relation to the foci of grantees, 
their spending of grant funds, and in terms of the technology resources acquired through the 
grant. To begin with, the foci of the grant programs were well aligned with the statewide Ed- 
Tech Program goals. Although each grant approached the statewide Ed-Tech goal of integrating 
technology with classroom instruction differently, all addressed the state goal. This is reflected in 
the wide variety of content areas addressed by the grants (with English language arts and science 
being the ones most frequently addressed). Moreover, most grant managers indicated that 
technology integration and the promotion of 21st century skills were the major goals of their 
grants. The grants frequently focused on professional development and teacher collaboration, 
both of which are strategies of the statewide Ed-Tech Program. Thus, it appears that the request 
for proposal process and subsequent monitoring of the programs was successful in maintaining 
the focus. 

The progress of grantees is also indicated by survey responses regarding the spending of grant 
funds. Grant managers typically report having spent the majority of grant funds overall, as well 
as those funds allocated to professional development. The few exceptions are grantees whose 
programs began during the 2010-11 school year. Grantees are acquiring the technology 
resources that are expected by the program. Most grantees reported that their expenditures on 
professional development were in the range required by the program (i.e., at least 25 percent of 
grant funds). 

Question 2: How Effectively Do Schools Support the Implementation 
of Project Goals? 

Professional development was considered by teachers to be of high quality and effective. The 
vast majority of teachers rated professional development as high quality, and nearly 75 percent 
of teachers across grant programs reported that their participation had a noticeable to very strong 
impact on their teaching practice. Teachers indicated that professional development was highly 
targeted and relevant. For example, most teachers indicated its major emphasis was using 
technology for instruction and learning. The professional development in these programs appears 
to have been sustained — that is, occurring over several months. 

eLearning Project teachers reported particularly high levels of impact on teaching practice. This 
finding may reflect the certain distinctive aspects of the eLearning Project relative to the other 
programs. First, the eLearning Project participants received a total of 6 to 10 days of professional 



American Institutes for Research 



2010-11 Final Report on Vermont’s Ed-Tech Program — 71 




development over the course of the year, which is twice as much as the next closest program (as 
described in question la). Second, the eLearning Project employed a team-based approach where 
collaboration was expected. Finally, each team addressed a specific school goal, which was 
identified with the input of the principal. These distinctive features of the eLearning Project 
reflect three research-based features of professional development: extended duration, collective 
participation, and coherence with school goals (Desimone, 2009). 

• Recommendation 1: Encourage effective professional development. Effective 
professional development was critical for the success of these grants. As indicated by 
teacher survey responses, the CBTG Program, the Educate/Innovate Program, and the 
eLearning Project were all successful in fostering changes in instructional practice. This 
success demonstrates the effectiveness of content-focused, standards-aligned professional 
development that focuses on the integration of technology. It is recommended that 
VTDOE continue to require a percentage of grant funds be devoted to professional 
development, and this professional development should focus strongly on standards- 
aligned integration of technology into instructional practice. 

The Ed-Tech Program enhanced collaboration, which in turn supported implementation of the 
grant goals. Grant managers and teachers indicated that teacher collaboration was a considerable 
focus of their Ed-Tech grants. The majority of teachers reported that their grant programs 
increased their level of collaboration with colleagues. They reported frequent collaboration on 
technology (e.g., sharing resources and troubleshooting) within grade-level teams or subject 
areas and, to a lesser extent, outside their grade level, subject, and school. In the interviews, 
teachers described both informal and formal opportunities to collaborate, occurring in regular 
grade-level or subject area meetings or during separate meetings arranged specifically for the 
purpose of the grant (e.g., the eLearning Project team meetings). Teachers reported that 
collaboration played a very important role in achieving the goals of their grant. 

• Recommendation 2: Provide opportunities for teacher collaboration. The findings 
from this evaluation underscore the importance of teacher collaboration. They suggest 
that administrators of technology integration programs should provide opportunities for 
teachers to collaboratively develop lessons, share resources, and troubleshoot. 
Administrators should set teacher collaboration as an expectation of the grant and ensure 
that teachers have sufficient opportunity for it. 

School leadership actively supported the grant goals with resources and technical support. 
Teachers reported high levels of principal awareness and encouragement of technology 
integration related to the grant. School leaders supported grants by providing resources (e.g., 
planning time) as well as leadership (e.g., setting expectations for technology integration). The 
extent of principal involvement varied somewhat within and across grant programs. Teachers 
from the eLearning Project reported the highest level of principal involvement. Apparently, the 
program’s requirement that the principal participate in team meetings was effective in promoting 
the principal’s involvement. In other programs, especially the Educate/Innovate Program, school 
and district technology leaders provided guidance and support. Teachers typically reported 
receiving sufficient technical support to accomplish the goals of their grant programs. 

• Recommendation 3: Encourage administrators to actively support implementation 
of technology grants. There are several ways for VTDOE to encourage principal 
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involvement. One of the most effective, based on the experience with the eLearning 
Project, is to require principals to participate in regular project team meetings. There are 
undoubtedly several other approaches to ensuring administrator support. For example, 
grant applications could require grantees to explain how a proposal is congruent with a 
school’s current goals for improvement and professional development. Professional 
development that is congruent with these goals is likely to be supported by the 
administration. Along similar lines, grant applications could ask grantees to explain their 
sources of technical support and have the principal verify that there are support staff who 
are able and willing to provide such support. 

Question 3: Do the Ed-Tech Grant Programs Promote Technology Integration 
in Support of Student-Centered Learning? 

The Ed-Tech Program appears to have promoted student-centered instruction. The majority of 
teachers report that their Ed-Tech grant programs had a noticeable to very strong impact on 
instructional practices, with the exception of LNV teachers. The programs promoted student- 
centered instruction in a variety of ways, including more project-based, collaborative, and 
personalized learning and greater differentiation of instruction. Across all programs, teachers 
noted that they assigned more projects than in the previous school year; here, too, the smallest 
impact was observed for LNV. For the CBTG Program, the Educate/Innovate Program, and the 
eLearning Project, it is reasonable to conclude that the professional development model was 
effective for promoting change in instructional practice. It is noteworthy that the professional 
development in each program emphasized alignment with Vermont learning standards or school 
goals and focused on integrating technology into instructional practice. LNV, according to the 
interviews, focused on technical mastery of the videoconferencing technology. It is reasonable to 
conclude that a strong focus on standards-aligned technology integration is critical for promoting 
change in practice. 

In the opinion of participants, the Ed-Tech Program increased the frequency of student 
technology use. Overall, teachers indicated that they are integrating technology into the 
classroom on a more frequent and consistent basis, particularly when compared to the previous 
school year. The frequency of student use of technology varied by program, but teachers reported 
frequent student use of technology for several purposes, including additional practice or skill 
reinforcement, online research, and paper writing. 

Question 4: What Are the Learning Outcomes of the Program 
in Terms of Student Engagement and Motivation and Mastery 
of Vermont Grade-Level Expectations? 

Teachers perceive that their Ed-Tech grant programs have increased student engagement and 
motivation. Most teachers reported that their Ed-Tech grant programs made a noticeable or very 
strong impact on student engagement, with CBTG teachers reporting the strongest impact. In the 
interviews, teachers noted several aspects of the program that promoted engagement and 
motivation: hands-on use of technology, project-based learning, and the opportunity to connect 
with outside experts and peers at other locations. 
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Teachers report that their Ed-Tech programs had an impact on content understanding but not on 
test perfonnance. Over three fifths of the teachers indicated that their Ed-Tech programs have 
increased students’ content understanding; this finding was particularly strong in the CBTG 
Program. This finding may reflect the program’s targeting of specific grade-level expectations. 

In the interviews, teachers attributed this perceived increase in content understanding to greater 
student engagement in tasks and more personalized learning (e.g., self-pacing and additional 
media for learning). Most teachers did not believe that the programs made a noticeable impact on 
student perfonnance on standardized tests. 

• Recommendation 4: Consider expanding the programs. In light of the impact on 
teaching practice and student learning, VTDOE should consider expanding these 
programs. This may involve plans for scaling up the grants at other schools so that the 
success of these programs can be shared and replicated. The successful features of these 
programs include their substantial investment in professional development, their focus on 
specific student learning goals aligned with state standards and school goals, and the 
opportunity for teacher collaboration. 

Question 5: To What Extent Are Changes in Teaching and Learning Adopted 
and Sustained? 

The majority of the Ed-Tech grant programs have plans for sustainability. Grant managers 
indicated that the majority of the Ed-Tech grant programs will be sustained. In most cases, 
additional local funding will be available to sustain the grant for the purchase of equipment, 
software, and additional or ongoing training. In some instances, such as with the LNV program, 
supplemental funding sources have not been secured, and the sustainability of the grant programs 
at sites without these additional funds is unclear. Across programs, grant managers reported that 
they plan to expand their grants to other teachers in the school and, in some cases, to teachers in 
other schools. In general, teachers who participated in the grant programs have gained the 
knowledge and skills necessary to help train other teachers on what they have learned. 

• Recommendation 5: Encourage grantees to plan for sustainability. To ensure that the 
work of the initial grant is sustained, VTDOE should require grantees to formally 
articulate their sustainability plans. These plans should address the securing of additional 
funds (if necessary) and the expansion of teacher participation. It is likely that securing 
the commitment of local resources to these grants will require community support. 
Therefore, VTDOE may also consider requiring grantees to articulate a strategy for 
parent and community outreach to gamer this support. 

Question 6: To What Extent Did the Program Support the Development and 
the Deployment of Online Learning Opportunities? 

The Ed-Tech Program funded the creation of the VTVLC program, which had 143 enrollments 
in 17 member schools during its first year. The strongest impact that principals noted from this 
program was on expanding the availability of courses, which is particularly important for the 
rural schools that are common in Vermont. Most principals were satisfied overall with the 
program, and most anticipated participating again in the following year. The courses did not 
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strongly incorporate student discussion as a learning tool, even though discussion forums are 
available in all the courses. 

• Recommendation 6. Promote student interactions through use of discussion forums. 

Student discussion is a hallmark of quality online teaching (International Association for 
K-12 Online Learning, 2010). Because the technical capability for student interaction 
already exists, this is a feasible opportunity for improving course quality. To promote the 
use of discussion forums, VTVLC program leaders should consider emphasizing the 
importance of student interaction in their teacher preparation course and monitoring 
courses to ensure that student interaction is occurring. 
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Appendix A. Survey Instruments 



Teacher Survey 

Thanks for agreeing to complete the Vennont Ed-Tech program’s Teacher Survey. You will be 
presented with five screens of questions. Please click the Submit button at the bottom of each screen to 
advance to the next screen. The survey should take about 15-20 minutes to complete; you should see a 
“Thank you” page when you are finished. 

1. Please select the Ed-Tech Grant Program in which you participated this school year. If you 
participated in more than one grant program, please complete a separate survey for each.: 

□ Content-Based Grants 

□ Educate Innovate 

□ eLearning 

□ Learning Network of V ermont 

□ Real World Design Challenge 

□ Vennont Virtual Learning Cooperative 

□ Other, please specify 

Lor the rest of the survey, all questions that refer to “your school’s Ed-Tech grant program” are referring 
to the program you have checked above. 

2. Which of the following represented a major goal of the grant program implemented at 
your school (select all that apply)? 

□ Improve students’ skills and/or achievement levels in a given content area 

□ Increase opportunities for project-based learning 

□ Technology Integration (across the school and/or across subjects) 

□ Increase opportunities for teachers to gain professional development regarding technology 
integration 

□ Promote 2 1 st Century skills 

□ Gain resources for the school (including equipment and professional development) 

Please answer the following section of questions based on your experiences with professional 
development related to the grant program you indicted above during the 2010-2011 school year 
(including the summer of 2010) . Do not consider other professional development events that you may 
have attended during the year that were not related to an Ed-Tech Grant Program. 

Attendance at Professional Development Events 

3. In which of the following professional development events did you participate this school 
year as part of your school’s Ed-Tech grant program (check all that apply): 

□ Attended an institute (i.e., a focused training or planning session spanning several days or 
longer) 

□ Worked onsite with a consultant (i.e., an outside expert who instructs and guides teachers in 
learning new processes and skills). 

□ Attended an in-school training (i.e., learning knowledge and skills from and with school 
faculty). 

□ Attended a workshop (i.e., an interactive session or sessions facilitated by an expert and 
focused on practical knowledge and skills). 

□ Attended off-site conferences provided by professional associations or organizations. 
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□ On-demand, online, or Web-delivered professional development. 

□ One-on-one or group training with technology coordinators or aides. 

□ Other. Please specify 

4. Approximately how many total hours did you participate in the events attended during the 

2010-2011 school year that you indicated above? (Enter a number 0 - 99, using numerals 
and not letters): 

Quality of professional development 

5. To what extent did attendance at the professional development events indicated above 
make an impact on your teaching practice? 

□ No impact 

□ Slight impact (to date) 

□ Noticeable impact 

□ Very strong impact 



6. How would you rate the professional development in terms of the following: 





Poor or 
Inadequate 


Barely 

Adequate 


Good 


Excellent 


Organization 










Clarity of content presented 










Connection to the goals of your Ed-Tech grant 










Provided resources and tools I can use in the 
classroom. 










Addressed the needs of the students in my classroom. 










Helped me to understand my role and responsibilities 
in implementing this program at my school. 











Level of Collaboration 



7. Over the past school year, how often have you collaborated with other teachers: 





Not at 

All 


1-2 times 
per 

Semester 


1-2 times 
per 
Month 


Weekly 


Daily/ 

Almost 

Daily 


Within your grade-level team or subject area 












Outside of your grade-level team or subject 
area, but within your school 












Outside of your school (e.g., through an 
online network) 













8. How important has collaboration with other teachers and school staff been for 
accomplishing the goals of your grant program? 

□ Not at All Important 

□ Minimally Important 

□ Moderately Important 

□ Very Important 

Student Purpose in Using Technology 

9. Over the past semester, how often did students in your classroom use technology for the 
following purposes during class time? 
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Daily/Almost 

daily 


Weekly 


1-2 times 
per month 


1-2 times 
per semester 


Not at 

all 


Additional practice or skill 
reinforcement 












Online communication 












Online research 












Analyzing Data 












Sorting/categorizing 

information 












Writing a paper 












Making a presentation 












Drawing or creating visual art 












Composing or editing music 












Creating or editing movies 













□ Other, please specify: 



Impact of program on teacher practice: Instruction and Inspiring student creativity 



10. Overall, how much of an impact has your Ed-Tech Grant Program had on: 





No 

impact 


Slight 
impact (to 
date) 


Noticeable 

impact 


Very 

strong 

impact 


Providing opportunities for students to explore real- 
world problems. 










Providing opportunities for students to work on 
extended projects (a week or more in duration). 










Providing opportunities for students to collaborate 










Incorporating digital tools and resources into lessons. 










Customizing learning activities to address students’ 
diverse ability levels. 










Providing opportunities for students to engage in 
interactive learning opportunities (in contrast to lecture- 
based instruction) 











11. This year, how many different projects (extending more than one week in duration) did 

you assign? (Enter a number, using numerals and not letters) 

12. During the 2009-2010 school year, approximately how many different projects (extending 

more than one week in duration) did you assign? 

Teacher Practice: Assessment 



13. Overall, how much of an impact has your Ed-Tech Grant Program had on: 





No 

impact 


Slight 
impact (to 
date) 


Noticeable 

impact 


Very 

strong 

impact 


Collecting various kinds of evidence of student 
learning. 










Designing assessments that are appropriate for 
project -based learning. 










Using student assessments to inform future lessons 










Using student assessments 
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Impact of program on teaching practice 



14. Overall, how much of an impact has your Ed-Tech Grant Program had on: 





No impact 
or negative 
impact 


Slight 
impact (to 
date) 


Noticeable 

impact 


Very 

strong 

impact 


Your effectiveness as a teacher 










Your enjoyment of teaching 










Your level of collaboration with colleagues 











Impact of program on student learning 

15. Overall, how much of an impact has your Ed-Tech Grant Program had on: 





No impact 
or negative 
impact 


Slight 
impact (to 
date) 


Noticeable 

impact 


Very 

strong 

impact 


Your students’ level of engagement 










Your students’ understanding of content 










Your students’ performance on standardized 
tests 










Your students’ ability to work collaboratively 











15a. Please explain the impact on student learning or engagement in your own words. 



Barriers and Challenges 

16. To what extent have the following posed as a barrier or challenge when implementing the 
grant program at your school? 





Did not pose 
a challenge 


Minimal 

Challenge 


Moderate 

Challenge 


Considerable 

Challenge 


Insufficient time to plan lessons 










Insufficient time to leam new technology 










Need to cover content 










Technology not working 
properly/ defective 










Limited support for technology 










Limited financial resources 










Lack of interest/resistance from teachers 










Internet: Slow connection speed, limited 
broadband available, no access at home 











Administrative Direction and Support 

17. Select the statement that best describes your principal’s support for the instructional goals 
of the Ed-Tech grant program: 

□ Principal does not understand or is not aware of the instructional practices aligned with the 
Ed-Tech program. 

□ Principal understands the instructional practices of the Ed-Tech program but does not 
actively encourage me to adopt these practices. 

□ Principal encourages me to adopt the instructional practices, but does not hold me 
accountable for them. 
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□ Principal both encourages me to adopt the instructional practices and holds me accountable 
for them. 

Teacher Background 

18. Including this year, how many years have you been a teacher? 

□ I am not a teacher. 

□ Less than one year 

□ 1 to 3 years 

□ 4 to 5 years 

□ 6 to 10 years 

□ 1 1 to 1 5 years 

□ More than 1 5 years 

19. How many years have you taught at this school? 

□ Less than one year 

□ 1 to 3 years 

□ 4 to 5 years 

□ 6 to 10 years 

□ 1 1 to 1 5 years 

□ More than 1 5 years 

20. What subject areas do you teach? Choose all that apply. 

□ Elementary education 

□ English language arts 

□ Mathematics 

□ Science 

□ Social studies (history, economics, civics, etc.) 

□ Foreign language 

□ Arts (music, drama, visual arts, dance) 

□ Physical and/or health education 

□ Other: 

21. What grade level do you teach? Choose all that apply. 

□ K - 3 

□ 4-6 

□ 7-8 

□ 9-12 

□ Other (Please explain): 

22. What is the name of your school? (drop down list) Note: To preserve confidentiality, (1) 
school data will not be reported if fewer than four people from a school respond, and (2) 
school data will not be broken down by grade or curriculum area for the same reason. 
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Grant Manager Survey 



Thanks for agreeing to complete the Vennont EETT program’s Grant Manager Survey. You will be 
presented with three screens of questions. Please click the Submit button at the bottom of each screen to 
advance to the next screen. The survey should take about 15-20 minutes to complete; you should see a 
“Thank you” page when you are finished. 

Program Information 

1. Name of the competitive EETT program under which your EETT grant was awarded (if 
there are more than one, complete a separate survey for each): 

□ Content-based grant 

□ eLearning Network 

□ Impacting Tobacco Prevention with Technology 

□ Learning Network of V ermont 

□ Vennont Virtual Learning Collaborative 

□ Other 

2. Areas of program focus (check all that apply) 

□ Mathematics 

□ Reading 

□ English/Language Arts 

□ Science 

□ Social Studies 

□ Arts 

□ PE/Health 

□ World Languages 

□ Technology Education (ITEA Standards) 

□ Technology Integration 

□ Technology Literacy 

□ Career Development 

□ Career and Technical Education 

□ Other (please specify) 

3. To what degree does your school’s EETT grant program focus on the following activities? 





Not a Focus 


Minor Focus 


Major Focus 


Professional development 








Teacher collaboration 








Purchase of equipment or software 








Installation of equipment or 
software 








Curriculum development 








Parental/community outreach 








Other (describe in the box below) 









If you selected “other” to the previous item, please describe: [text box] 

4. Total grant amount (in dollars): 

□ $0 - $ 10,000 
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□ $ 10,001 - 20,000 

□ $20,001 -$30,000 

□ $30,001 -$40,000 

□ $40,001 -50,000 

□ Greater than $50,000 

5. Approximately what proportion of your grant amount have you expended to date? 

□ 0 - 20 % 

□ 41-60% 

□ 61-80% 

□ 81-100% 

□ Unsure 



6, Approximately what percentage of grant funds has been spent on the following? 





None 


1-19% 


20-39% 


40-59% 


60-79% 


80-99% 


Connectivity/Networking 














Curriculum development 














Desktop computers 














Evaluation 














Handheld devices 














Interactive white boards 














Laptop computers 














Professional development 














Software/ courseware 














T echnology integrator/coach/mentor 














Virtual learning course content 














Virtual learning management system 














Visual or audio equipment 














Website development 














Other 















If you selected “other” to the previous item, please describe: [text box] 

7. What percentage of the funds allocated to professional development has been spent so far? 

□ 0 - 20 % 

□ 41-60% 

□ 61-80% 

□ 81-100% 

□ Unsure 

Professional Development 

8. What has the professional provided to date emphasized? 





Not an Emphasis 


Minor Emphasis 


Major Emphasis 


2 1 st century skills 








Project-based learning 
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Uses of technology for instruction 
and learning 








Student assessment 








Inquiry-based instruction 








Experiential learning 








Team teaching 








Differentiated instruction 








Other (describe in the box below) 









If you selected “other” to the previous item, please describe: [text box] 

9. What kinds of professional development opportunities have been offered or made possible 
by the program to date (check all that apply): 

□ Institute (i.e., a focused training or planning session spanning several days or longer) 

□ Onsite session with a consultant (i.e., an outside expert who instructs and guides teachers in 
learning new processes and skills) 

□ In-school training (i.e., learning knowledge and skills from and with school faculty) 

□ Workshop (i.e., an interactive session or sessions facilitated by an expert and focused on 
practical knowledge and skills) 

□ Off-site conference provided by professional associations or organizations 

□ Network of teachers outside the school or district to discuss implementation 

□ On-demand, online, or Web-delivered professional development 

□ One-on-one or group training with technology coordinators or aides 

□ Other 

10. What is the total estimated number of hours of professional development provided to date 
per participant ? 

□ None - professional development activities have not yet started 

□ 1-5 hours 

□ 6-10 hours 

□ 11-15 hours 

□ 16-20 hours 

□ 21-25 hours 

□ 26-30 hours 

□ 31 hours or more 

11. What is the total number of individuals who have participated in professional development 
activities to date? [response blank] 

Program Status 



12. What will the program focus on over the summer recess? 





Not a Focus 


Minor 

Focus 


Moderate 

Focus 


Major 

Focus 


Professional development 










Teacher collaboration 










Purchase of equipment or software 










Installation of equipment or 
software 










Curriculum development 










Parental/community outreach 
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Other (describe in the box below) 











If you selected “other” to the previous item, please describe: [text box] 

13. What is the current stage of program implementation to meet the goals of your grant? 

□ Program Installation - structural supports, strategies, policies and procedures are put in place 

□ Early Implementation - expectations for the program are high but new skills and appropriate 
support systems are still fragile and developing 

□ Full Implementation - program is fully operational 

□ Modification - program is undergoing refinement based on feedback or assessment 

□ Sustainability - focus on the long-term survival and continued effectiveness of the program 

□ Unsure 

14. How many additional staff members from your school do you expect to participate in the 
program next year? 

□ None 

□ 1-3 

□ 4-6 

□ 7-9 

□ 10 or more 

□ Unsure 

15. When do you anticipate grant funding to be fully expended? (month) (year) 

16. Will local funding be available to continue the program once grant funding is finished? 

□ Yes 

□ No 

□ Don’t know 

17. What have been the major facilitators of your grant program’s progress or success? 

18. What have been the major barriers that the grant program has faced to date? 
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VTVLC Principal Survey 



Thanks for agreeing to complete the Vennont Ed-Tech program’s Teacher Survey. You will be 
presented with five screens of questions. Please click the Submit button at the bottom of each screen to 
advance to the next screen. The survey should take about 15-20 minutes to complete; you should see a 
“Thank you” page when you are finished. 

1. Please indicate what phase your school is in: 

□ Phase I 

□ Phase II 

Satisfaction 

2. What is your overall level of satisfaction with the VTVLC program? 

□ Not Satisfied 

□ Somewhat Satisfied 

□ Moderately Satisfied 

□ Very Satisfied 

3. In your opinion, how satisfied are your students with the VTVLC program? 

□ Not Satisfied 

□ Somewhat Satisfied 

□ Moderately Satisfied 

□ Very Satisfied 

□ Don’t know/not sure 

Impact 



4. To what degree has the VTVLC program made an impact on: 





No Impact 


Slight 

Impact 


Noticeable 

Impact 


Very Strong 
Impact 


students’ opportunities to enroll in courses 
that would not have been available otherwise 










the flexibility of students’ schedules 










students’ opportunities to enroll in advanced 
courses 










school’s scheduling flexibility 











Benefits 

5. How have your students benefited from the VTVLC program (check all that apply)? 

□ Learned to work independently 

□ Learned to work in an online community 

□ Learned to take personal responsibility for their education 

□ Improved their ability to effectively use technology for their education 

□ Other (please specify)? 

6. How have your teachers benefited from the VTVLC program (check all that apply)? 

□ Improved their ability to effectively use technology for instruction 

□ Learned to teach in an online environment 

□ Improved their ability to use technology to communicate 
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□ Other (please specify)? 

7. Apart from VTVLC, what other online learning provider has your school used (if any)? 

□ Name of provider: 



8. How does the VTVLC program compare with the provider you named above (Note: if your 
school is in Phase II, please skip this question): 





VTVLC 
was much 
less 

effective 


VTVLC was 
somewhat 
less effective 


VTVLC 
was about 
as effective 


VTVLC was 
somewhat 
more effective 


VTVLC was 
much more 
effective 


Communication with schools 












Problem resolution 












Responsiveness to school needs 












Responsiveness student needs 












Access to courses that our 
students want 












Quality of courses 













Barriers 

9. Which of the following are major barriers to your school’s continued participation in the 
VTVLC program (check all that apply)? 

□ Cost of program 

□ Low student interest 

□ Lack of parental involvement 

□ Lack of district support 

Sustainability 

10. How likely is your school to continue participating in the VTVLC program in two years? 

□ 0% - We will certainly not participate 

□ 25% - We probably won’t participate 

□ 50% - Unsure if we will or won’t 

□ 75% - We probably will participate 

□ 100% - We will certainly participate 

11. How should the VTVLC program be improved? 

12. Would you be interested in participating in a follow-up phone interview about your 
responses to this survey? 

□ Yes 

□ No 
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Appendix B.Teacher Survey Tables 



Table B-l. Quality of Professional Development by Grant Program 



How would you rate the professional 
development in terms of the following: 


Overall 

N=158 


CBTG 

n=12 


Educate/ 

Innovate 

n=46 


eLearning 

«=17 


LNV 

n=17 


Organization 


Poor or Inadequate 


1 . 3 % 


0.0% 


0.0% 


0.0% 


0.0% 


Barely Adequate 


3 . 8 % 


2.8% 


2.2% 


17.6% 


0.0% 


Good 


64 . 6 % 


65.3% 


60.9% 


70.6% 


70.6% 


Excellent 


30 . 4 % 


31.9% 


37.0% 


11.8% 


29.4% 


Clarity of content 
presented 


Poor or Inadequate 


1 . 3 % 


0.0% 


0.0% 


0.0% 


0.0% 


Barely Adequate 


1 . 9 % 


2.8% 


0.0% 


0.0% 


5.9% 


Good 


60 . 8 % 


59.7% 


54.3% 


88.2% 


58.8% 


Excellent 


36 . 1 % 


37.5% 


45.7% 


11.8% 


35.3% 


Connection to the 
goals of your Ed- 
Tech grant 


Poor or Inadequate 


2 . 5 % 


1.4% 


2.2% 


0.0% 


0.0% 


Barely Adequate 


5 . 1 % 


7.0% 


6.5% 


0.0% 


0.0% 


Good 


49 . 0 % 


46.5% 


41.3% 


76.5% 


58.8% 


Excellent 


43 . 3 % 


45.1% 


50.0% 


23.5% 


41.2% 


Provided resources 
and tools 1 can use 
in the classroom. 


Poor or Inadequate 


1 . 9 % 


0.0% 


2.2% 


0.0% 


0.0% 


Barely Adequate 


3 . 8 % 


2.8% 


4.3% 


0.0% 


11.8% 


Good 


44 . 9 % 


45.8% 


34.8% 


70.6% 


47.1% 


Excellent 


49 . 4 % 


51.4% 


58.7% 


29.4% 


41.2% 


Addressed the needs 
of the students in 
my classroom. 


Poor or Inadequate 


1 . 9 % 


1.4% 


0.0% 


0.0% 


0.0% 


Barely Adequate 


7 . 6 % 


5.6% 


10.9% 


5.9% 


11.8% 


Good 


63 . 3 % 


61.1% 


60.9% 


76.5% 


70.6% 


Excellent 


27 . 2 % 


31.9% 


28.3% 


17.6% 


17.6% 


Helped me to 
understand my role 
and responsibilities 
in implementing 
this program at my 
school. 


Poor or Inadequate 


3 . 8 % 


2.8% 


0.0% 


0.0% 


11.8% 


Barely Adequate 


10 . 1 % 


11.1% 


10.9% 


5.9% 


11.8% 


Good 


55 . 1 % 


51.4% 


56.5% 


76.5% 


47.1% 


Excellent 


31 . 0 % 


34.7% 


32.6% 


17.6% 


29.4% 



Note: Not all columns sum to 100.0% due to rounding. Overall column includes up to 3 respondents from RWDC 
and 3 whose program was not identified. 
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Table B-2. Level of Collaboration by Program 



Over the past school year, how often 
have you collaborated with other 
teachers: 


Overall 


CBTG 


Educate/ 

Innovate 


eLearning 


LNV 


Within your grade- 
level team or 
subject area 


N 


149 


68 


45 


16 


14 


Not at All 


4.0% 


4.4% 


2.2% 


0.0% 


7.1% 


2 to 3 Times per 
Semester 


10.7% 


10.3% 


13.3% 


6.3% 


14.3% 


Monthly 


16.1% 


16.2% 


20.0% 


6.3% 


14.3% 


Weekly 


30.9% 


25.0% 


35.6% 


43.8% 


21.4% 


Daily/Almost Daily 


38.3% 


44.1% 


28.9% 


43.8% 


42.9% 


Outside of your 
grade-level team or 
subject area, but 
within your school 


N 


148 


67 


45 


16 


14 


Not at All 


10.1% 


10.4% 


11.1% 


6.3% 


0.0% 


2 to 3 Times per 
Semester 


26.4% 


28.4% 


22.2% 


31.3% 


35.7% 


Monthly 


25.7% 


16.4% 


35.6% 


31.3% 


21.4% 


Weekly 


27.7% 


31.3% 


26.7% 


25.0% 


21.4% 


Daily/Almost Daily 


10.1% 


13.4% 


4.4% 


6.3% 


21.4% 


Outside of your 
school (e.g., 
through an online 
network) 


N 


146 


67 


43 


16 


14 


Not at All 


32.9%o 


35.8% 


27.9% 


50.0% 


7.1% 


2 to 3 Times per 
Semester 


34.9% 


35.8% 


41.9% 


18.8% 


35.7% 


Monthly 


19.2% 


17.9% 


18.6% 


25.0% 


14.3% 


Weekly 


11.6% 


9.0% 


11.6% 


6.3% 


35.7% 


Daily/Almost Daily 


1.4% 


1.5% 


0.0% 


0.0% 


7.1% 



Note: Overall column includes up to 3 respondents from RWDC and 3 whose program was not identified. 
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Table B-3. Teacher Ratings of Frequency of Student Use of Technology for Different Purposes, by Program 



Over the past semesti 
students in your class 
for the following pur] 


er, how often did 
;room use technology 
poses during class time? 


Overall 


CBTG 


Educate/ 

Innovate 


eLearning 


LNV 


Additional practice 
or skill 

reinforcement 


N 


147 


67 


45 


15 


14 


Not at All 


6.8% 


4.5% 


8.9% 


0.0% 


15.4% 


2-3 Times per Semester 


15.0% 


14.9% 


17.8% 


12.5% 


15.4% 


Monthly 


18.4% 


16.4% 


17.8% 


25.0% 


15.4% 


Weekly 


34.0% 


34.3% 


33.3% 


37.5% 


30.8% 


Daily/Almost Daily 


25.9% 


29.9% 


22.2% 


25.0% 


23.1% 


Online 

communication 


N 


149 


68 


45 


16 


14 


Not at All 


28.2% 


42.6% 


17.8% 


5.9% 


15.4% 


2-3 Times per Semester 


14.1% 


5.9% 


22.2% 


29.4% 


15.4% 


Monthly 


20.8% 


23.5% 


15.6% 


23.5% 


23.1% 


Weekly 


24.2% 


22.1% 


24.4% 


29.4% 


30.8% 


Daily/Almost Daily 


12.8% 


5.9% 


20.0% 


11.8% 


15.4% 


Online research 


N 


149 


68 


45 


16 


14 


Not at All 


14.1% 


25.0% 


2.2% 


5.9% 


0.0% 


2-3 Times per Semester 


15.4% 


13.2% 


22.2% 


11.8% 


15.4% 


Monthly 


21.5% 


25.0% 


20.0% 


11.8% 


23.1% 


Weekly 


32.2% 


32.4% 


26.7% 


47.1% 


30.8% 


Daily/Almost Daily 


16.8% 


4.4% 


28.9% 


23.5% 


30.8% 


Analyzing Data 


N 


149 


68 


45 


16 


14 


Not at All 


35.6% 


42.6% 


31.1% 


17.6% 


38.5% 


2-3 Times per Semester 


20.1% 


20.6% 


15.6% 


23.5% 


23.1% 


Monthly 


27.5% 


22.1% 


40.0% 


29.4% 


15.4% 


Weekly 


12.8% 


11.8% 


11.1% 


29.4% 


7.7% 


Daily/Almost Daily 


4.0% 


2.9% 


2.2% 


0.0% 


15.4% 


Sorting/categorizing 

information 


N 


149 


68 


45 


16 


14 


Not at All 


28.2%o 


35.3% 


20.0% 


29.4% 


23.1% 


2-3 Times per Semester 


24.8% 


23.5% 


26.7% 


23.5% 


15.4% 


Monthly 


28.9% 


25.0% 


40.0% 


23.5% 


23.1% 


Weekly 


12.8% 


11.8% 


11.1% 


23.5% 


15.4% 


Daily/Almost Daily 


5.4% 


4.4% 


2.2% 


0.0% 


23.1% 
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Writing a paper 


N 


149 


68 


45 


16 


14 


Not at All 


22.1% 


36.8% 


4.4% 


17.6% 


7.7% 


2-3 Times per Semester 


16.1% 


14.7% 


22.2% 


5.9% 


23.1% 


Monthly 


22.8% 


17.6% 


26.7% 


23.5% 


30.8% 


Weekly 


24.8% 


22.1% 


26.7% 


47.1% 


15.4% 


Daily/Almost Daily 


14.1% 


8.8% 


20.0% 


5.9% 


23.1% 


Making a 
presentation 


N 


147 


67 


45 


15 


14 


Not at All 


22.4% 


35.3% 


11.4% 


0.0% 


7.7% 


2-3 Times per Semester 


22.4%, 


17.6% 


27.3% 


25.0% 


30.8% 


Monthly 


35.4% 


32.4% 


31.8% 


56.3% 


38.5% 


Weekly 


14.3% 


10.3% 


25.0% 


18.8% 


0.0% 


Daily/Almost Daily 


5.4% 


4.4% 


4.5% 


0.0% 


23.1% 


Drawing or creating 
visual art 


N 


149 


68 


45 


16 


14 


Not at All 


37.6% 


41.2% 


28.9% 


41.2% 


38.5% 


2-3 Times per Semester 


24.2%o 


20.6% 


28.9% 


29.4% 


15.4% 


Monthly 


22.8% 


25.0% 


28.9% 


11.8% 


7.7% 


Weekly 


10.7% 


10.3% 


8.9% 


17.6% 


15.4% 


Daily/Almost Daily 


4.7% 


2.9% 


4.4% 


0.0% 


23.1% 


Composing or 
editing music 


N 


149 


68 


45 


16 


14 


Not at All 


74.5%, 


72.1% 


77.8% 


70.6% 


76.9% 


2-3 Times per Semester 


12.8 % 


11.8% 


15.6% 


11.8% 


15.4% 


Monthly 


6.0% 


5.9% 


2.2% 


17.6% 


7.7% 


Weekly 


4.0% 


4.4% 


4.4% 


0.0% 


0.0% 


Daily/Almost Daily 


2.7% 


5.9% 


0.0% 


0.0% 


0.0% 


Creating or editing 
movies 


N 


149 


68 


45 


16 


14 


Not at All 


51.7% 


57.4% 


51.1% 


17.6% 


53.8% 


2-3 Times per Semester 


29.5% 


25.0% 


31.1% 


52.9% 


23.1% 


Monthly 


12.8% 


7.4% 


15.6% 


29.4% 


15.4% 


Weekly 


5.4% 


8.8% 


2.2% 


0.0% 


7.7% 


Daily/Almost Daily 


0.7% 


1.5% 


0.0% 


0.0% 


0.0% 



Note: Overall column includes up to 3 respondents from RWDC and 3 whose program was not identified. 
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Table B-5. Teacher Ratings of Program Impact on Teaching Effectiveness and Enjoyment, by Grant 

Program 



Overall, how much of an impact has your 
Ed-Tech grant program had on: 


Overall 


CBTG 


Educate/ 

Innovate 


eLearning 


LNV 


Your 

effectiveness as 
a teacher 


N 


146 


67 


44 


17 


12 


No Impact/Negative Impact 


4.1% 


4.5% 


0.0% 


0.0% 


8.3% 


Slight Impact (to date) 


25.3% 


20.9% 


27.3% 


17.6% 


41.7% 


Noticeable Impact 


49.3% 


52.2% 


54.5% 


58.8% 


25.0% 


Very Strong Impact 


21.2% 


22.4% 


18.2% 


23.5% 


25.0% 


Your 

enjoyment of 
teaching 


N 


145 


67 


43 


17 


12 


No Impact/Negative Impact 


6.9%> 


3.0% 


7.0% 


0.0% 


25.0% 


Slight Impact (to date) 


23.4%o 


20.9% 


25.6% 


29.4% 


25.0% 


Noticeable Impact 


42.1%, 


43.3% 


41.9% 


41.2% 


41.7% 


Very Strong Impact 


27.6%, 


32.8% 


25.6% 


29.4% 


8.3% 



Note: Overall column includes up to 3 respondents from RWDC and 3 whose program was not identified. 



Table B-6. Teacher Rating of Program Impact on Student Skills, by Grant Program 



Overall, how much of an impact has your 
Ed-Tech grant program had on: 


Overall 


CBTG 


Educate/ 

Innovate 


eLearning 


LNV 


Your students’ 
understanding of 
content 


N 


140 


63 


42 


17 


12 


No Impact/ Negative Impact 


8.6% 


1.6% 


7.1% 


11.8% 


33.3% 


Slight Impact (to date) 


28.6% 


19.0% 


45.2% 


23.5% 


33.3% 


Noticeable Impact 


50.7% 


60.3% 


42.9% 


58.8% 


33.3% 


Very Strong Impact 


12.1% 


19.0% 


4.8% 


5.9% 


0.0% 


Your students’ 
performance on 
standardized 
tests 


N 


118 


54 


38 


11 


11 


No Impact/ Negative Impact 


61.0% 


61.1% 


52.6% 


54.5% 


81.8% 


Slight Impact (to date) 


28.8% 


27.8% 


36.8% 


27.3% 


18.2% 


Noticeable Impact 


9.3% 


9.3% 


10.5% 


18.2% 


0.0% 


Very Strong Impact 


0.8% 


1.9% 


0.0% 


0.0% 


0.0% 


Your students’ 
ability to work 
collaboratively 


N 


139 


62 


42 


17 


12 


No Impact/Negative Impact 


13.7% 


17.7% 


7.1% 


5.9% 


25.0% 


Slight Impact (to date) 


27.3% 


24.2% 


26.2% 


29.4% 


50.0% 


Noticeable Impact 


39.6% 


32.3% 


50.0% 


52.9% 


25.0% 


Very Strong Impact 


19.4% 


25.8% 


16.7% 


11.8% 


0.0% 



Note: Category columns may not sum to 100.0% due to rounding. Overall column includes up to 3 respondents from RWDC 
and 3 whose program was not identified. 
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